Technical

IXP Manager

MICE uses IXP Manager. Login at: https://ixpmgr.micemn.net

The looking glass does not require a login.

Member Port Summary

  • Spanning tree shall be disabled (no BPDUs)
  • Proxy ARP shall be disabled. Use "no ip proxy-arp" or similar.
  • An MTU of 9000 is supported
  • Multicasts, Broadcasts, and Unknown Unicasts are limited to 100M
  • A maximum of 1 incoming MAC address. Due to Zayo’s non-compliance with MICE’s MAC limiting policy, MAC filtering is in place for their remote switch; participants on the Zayo remote switch must inform MICE of their specific MAC address upon initial connection and if it changes.
  • For recommendations on how to configure your equipment, see the excellent AMS-IX Config Guide or the "Any other tips on configuring my router?" question in the SeattleIX FAQ.
  • Use of the route servers is optional.
  • Note that your BGP sessions to the route servers likely need no bgp enforce-first-as or similar. The route servers do not add the MICE AS to the AS path.
  • BFD (Bidirectional Forwarding Detection) is optional. See the route server BFD section below for more details on route server BFD. BFD on bilateral sessions is between members to negotiate.

Route Servers

  • Route Server 1: 206.108.255.1 / 2001:504:27::d1af:0:1
  • Route Server 2: 206.108.255.2 / 2001:504:27::d1af:0:2
  • Note that your BGP sessions to the route servers likely need no bgp enforce-first-as or similar. The route servers do not add the MICE AS to the AS path.
  • The route servers are configured using IXP Manager. Various filters are applied.
  • The route servers enforce a max-prefix limit. This is copied from PeeringDB, if your AS exists in PeeringDB. Otherwise, the IXP Manager default is 250.

IRR

The route servers require participants to document their routes and downstream ASNs in IRR objects. If you are not familiar with this process, we recommend you use ARIN’s service (assuming you are in the ARIN region). They have documentation available.

  1. Start by creating an as-set object with your ASN and each downstream’s ASN (or their as-set, if they also have downstream ASNs).
  2. Create route/route6 objects for each route (prefix) you will originate.
  3. Your downstream ASNs will also need to create route/route6 objects (or you will have to create them on their behalf in an IRR like RADB).
  4. Reference your as-set in PeeringDB. If you do not list it in PeeringDB, then you must inform MICE of your as-set via email to peering@micemn.net.

BFD

The route servers are configured for passive BFD, meaning they will use BFD with you, but you need to start it. That is, you must be configured for active BFD. The route servers are configured with a minimum interval of 500 ms and a multiplier of 3.

BGP Communities

The default behavior of the MICE route servers is to announce all routes to all peers. You may use the typical communities (but pick one or the other; do not mix) to modify the announcements:

Description Community Large Community
Prevent announcement of a prefix to a peer 0:peer-as 53679:0:peer-as
Announce a route to a certain peer 53679:peer-as 53679:1:peer-as
Prevent announcement of a prefix to all peers 0:53679 53679:0:0
Announce a route to all peers (the default) 53679:53679 53679:1:0

For example, setting 0:1234 0:5678 (or 53679:0:1234 53679:0:5678) or causes the route server to announce the route to everyone except AS1234 and AS5678. This is an opt-out model.

To use an opt-in model, set 0:53679 (or 53679:0:0) plus the communities for each AS you want to receive the route. For example, setting 0:53679 53679:1234 53679:5678 (or 53679:0:0 53679:1:1234 53679:1:5678) causes the route servers to announce the route to only AS1234 and AS5678. If you choose an opt-in model, you may want to monitor the MICE-DISCUSS mailing list to hear about new members connecting.

Equipment

The MICE Equipment (.odg) diagram shows how the various switches are connected.

Here are some pictures of the rack, from 2017-05-10: full cabinet, switch, top of cabinet.

MICE Arista Switch Configuration

This shows how the MICE side is configured and is for reference only. While some of this may be applicable to the member side, it is not a set of instructions on how to configure your device. For that, see the AMS-IX Config Guide or the "Any other tips on configuring my router?" question in the SeattleIX FAQ. 

errdisable recovery cause bpduguard
errdisable recovery cause link-flap
errdisable recovery cause portsec
no lldp run
platform fap voq scheduling round-robin
platform sand lag hardware-only
spanning-tree mode rapid-pvst
spanning-tree portfast bpduguard default
spanning-tree portfast bpdufilter default
interface EthernetX/X
  mtu 9214
  switchport port-security maximum 1
  switchport port-security
  storm-control broadcast level 1
  storm-control multicast level 1
  spanning-tree portfast

MICE Juniper Switch Configuration

This shows how the MICE side is configured and is for reference only. While some of this may be applicable to the member side, it is not a set of instructions on how to configure your device. For that, see the AMS-IX Config Guide or the "Any other tips on configuring my router?" question in the SeattleIX FAQ. 

set interface X/X/X mtu 9216
set protocols rstp interface X/X/X edge
set ethernet-switching-options bpdu-block interface X/X/X
set ethernet-switching-options secure-access-port interface X/X/X mac-limit 1
set ethernet-switching-options storm-control interface X/X/X bandwidth 100000
set ethernet-switching-options bpdu-block disable-timeout 60
set ethernet-switching-options storm-control interface all

MICE Cisco Switch Configuration

This shows how the MICE side is configured and is for reference only. While some of this may be applicable to the member side, it is not a set of instructions on how to configure your device. For that, see the AMS-IX Config Guide or the "Any other tips on configuring my router?" question in the SeattleIX FAQ. 

system mtu jumbo 9198
mac address-table aging-time 14400
errdisable detect cause link-flap
errdisable recovery cause link-flap
errdisable recovery cause storm-control
vtp mode transparent
switchport block multicast
switchport block unicast
switchport port-security maximum 1
switchport port-security
switchport port-security violation restrict
storm-control broadcast level 20.00
spanning-tree bpdufilter enable
no cdp enable

Remote Switches

If you are interested in operating a MICE remote switch, please contact peering@micemn.net.

The process for connecting a new remote switch is as follows:

  1. The remote switch operator will provide a proposal to the board addressing both technical and business details. Switches dedicated to MICE are quasi-required. Proposals should address expected participation. For example, does the operator have firm commitments from e.g. 5 participants?
  2. The board will publish the proposal to the discussion list.
  3. After a reasonable comment period, the board will approve or deny the proposal.

Remote switch operators’ obligations include (but are not necessarily limited to):

  • Operators must obtain prior approval from the board for modifications.
  • Operators are responsible for the costs of operating their remote switch and the links to the core switch. They must monitor their traffic levels and promptly add capacity to keep the links running congestion-free.
  • Operators must enforce MICE’s technical port rules on their remote switches.
  • Operators must coordinate participant connections and disconnections with MICE. MICE allocates exchange IP addresses and documents participant connections.
  • Operators must inform their participants that the participants are not connecting directly to MICE. The remote switch operator cannot claim to be MICE.
  • Operators must inform their participants that the participants are also subject to MICE rules, procedures, and costs.