Jade's Tech Notes

More bits more places

Inside the Meraki MR58

We were recently gifted a box of old Meraki MR58s from a friendly neighborhood consultant that had removed them from service. Since there is essentially no information available on using the Meraki MR58 outside of the Meraki environment I had little hope that we’d be able to put them to good use.

Anyway, here I am a few months later sitting at home looking for a puzzle to solve while trying to get over a cold…

Replacing a Failed Disk in SmartOS

Replacement drives are not automatically discovered in SmartOS. Here’s how you replace a drive without rebooting the server.

This assumes the failed drive is c0t0d0 in slot sata0/0

Replacing a failed drive

  1. Offline the failed drive: zpool offline zones c0t0d0
  2. devfsadm -C -c disk -v (may not be necessary)
  3. devfsadm -c disk -v (may not be necessary)
  4. cfgadm -l you should see type sata-port & receptacle disconnected. If you don’t do this, type will be unknown (may not be necessary)
  5. cfgadm -f -c connect sata0/0
  6. cfgadm -l should show type disk, receptacle connected, occupant unconfigured
  7. cfgadm -c configure sata0/0
  8. cfgadm -l should now show occupant configured
  9. zpool replace zones c0t0d0 I got an error that the drive was already part of a ZFS pool because it was a used drive. In that case, do: zpool replace -f zones c0t0d0

When things go wrong

Your replacement drive might not work. My first replacement wouldn’t spin up. This section goes through what that looked like.

You run zpool replace zones c0t0d0 and get this message: cannot label 'c0t0d0': try using fdisk(1M) and then provide a specific slice

So you try to run fdisk to label the drive but it fails:

1
2
fdisk /dev/dsk/c0t0d0
fdisk: Cannot stat device /dev/dsk/c0t0d0

Check for device probe errors: dmesg | grep disk@0,0|tail -1 and find a message that it failed to power up: 2016-07-13T18:33:16.997975+00:00 c8-0a-a9-57-4c-ee genunix: [ID 353554 kern.warning] WARNING: Device /pci@0,0/pci152d,8975@1f,2/disk@0,0 failed to power up.

So, the drive is bad. Replace the drive and start over. TL;DR:

1
2
3
4
5
6
cfgadm -f -c connect sata0/0
dmesg | grep disk@3,0|tail -1
cfgadm -l
cfgadm -c configure sata0/3::dsk/c0t0d0
cfgadm -l
dmesg | grep disk@0,0|tail -1

Ubiquiti AirFiber 5X: The Perfect Medium-capacity Backhaul

Ubiquiti is going to announce their AirFiber 5X radio tomorrow at the Animal Farm User Group (AFMUG) meeting in Salt Lake City tomorrow. The quick start guide was posted early to their web site and gives us a sneak peak. From their teaser emails (that mention price and nothing else) it has a MSRP of $399 per radio (without dishes).

Essentially it’s a connectorized version of the AirFiber 5. It uses airFiber branded dishes or RocketDishes with a retrofit kit. Dishes are cross-pol instead of V/H. It supports channel sizes from 10Mhz to 50Mhz.

I’m pretty excited about this radio. It fits a niche between the Rocket & ePMP radios on the low end and licensed microwave on the high end. This is a market where Cambium’s PTP3/5/600 radios have reigned supreme but at a significantly higher price point. The ability to use large dishes means it has a fair chance to actually hit the marketing numbers of 500Mbps HDX or 200Km range.

Also, it doesn’t hurt that this is coming from their AirFiber group out of Chicago. They seem to have a better track record of delivering products with fewer teething problems relative to the AirMax M/Ti/AC line. I assume that’s because their people brought along some of that Motorola engineering discipline when they left the Canopy group. In any case, I expect this to be a radio that I can drop in to replace a couple wonky AirMax ac links and use instead of upgrading to PTP500 in some other cases and have it actually work without fuss.

Ubiquiti AirFiber24HD: Solid Improvement With Excessive Hype

Ubiquiti announced their AirFiber24HD today. It looks like a solid incremental improvement to the classic AirFiber24 with a roughly 250 Mbps (FDX) increase in throughput.

Key differences

Feature AF24 AF24HD
Maximum Modulation:   64QAM   256QAM
RX Gain: 38 dBi   40 dBi
Housing: Plastic Aluminum
Maximum Throughput (FDX):    774 Mbps   1Gbps
Link MSRP: $3,000 $6,000

Excessive hype

A unlicensed-frequency radio that can do bog standard 1Gbps FDX is actually a big deal. You can use these for short links between buildings and achieve just as much throughput as if you trenched fiber and used normal SFPs. No more caveats about actually only having 700Mbps of throughput.

Unfortunately with Ubiquiti a decent improvement isn’t good enough. Every product or service has to be a revolutionary disruptive breakthrough for the masses. This was on full display with the AF24HD launch where apparently line-rate 1G wasn’t good enough.

That’s right. The headline throughput number for a 1Gbps FDX (2Gbps HDX/aggregate) radio is shown as 20 Gbps because that’s the theoretical aggregate throughput of devices that may be connected behind the link. If that was a thing I’d be bragging about my network having over 3Tbps of throughput, except we don’t because that would be dishonest.

Cisco Site-to-site VPN With Outside VRF

TL;DR: Outside interface is a VRF, inside on global, must Specify VRF on crypto keychain.

I recently set up a IPSec VPN between two Cisco routers where the Internet-facing interface was in a VRF and the tunnel interface was in not in a VRF.

The VTI was configured in the usual way:

VTI Configuration
1
2
3
4
5
6
7
8
9
10
11
interface Tunnel0
 description VPN: site2 via provider A
 ip address 192.168.1.1 255.255.255.254
 ip mtu 1438
 keepalive 5 3
 tunnel source 192.0.2.1
 tunnel mode ipsec ipv4
 tunnel destination 192.0.2.2
 tunnel vrf Outside-A
 tunnel protection ipsec profile vpn-to-site2-via-provider-a
!

With the usual crypto configuration:

IPSec configuration
1
2
3
4
5
6
7
8
9
crypto isakmp policy 1
 encr aes
 authentication pre-share
 group 2
 lifetime 3600
crypto isakmp key SecretKey address 192.0.2.2
crypto ipsec transform-set vpn-peer esp-aes esp-sha-hmac
crypto ipsec profile vpn-to-site2-via-provider-a
 set transform-set vpn-peer

but the tunnel wouldn’t come up.

“debug crypto isakmp” showed a error to the effect that a preshared key wasn’t configured or couldn’t be found for the peer. This was puzzling because there was clearly a preshared key defined.

There weren’t any messages in the debug logs about incoming connections or policy mismatches so I assumed that the problem was occurring before the router tried to communicate with its peer or it wasn’t seeing traffic from the peer.

From the documentation I found it seemed like the “tunnel vrf Outside-A” statement in the VTI was all that was necessary for IOS to know that the outside was in a VRF (the tunnel endpoint is in a VRF so the protection profile must be on the VRF as well). This turned out to not be the case. The vrf statement on the VTI is just for the tunnel. You need to specify the VRF in the crypto keyring definition otherwise the router isn’t looking for VPN traffic in the VRF.

I changed the crypto configuration to use a keyring and specified the outside vrf on the keyring definition:

Working IPSec configuration (VRF keychain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
crypto keyring vpn-peer vrf Outside-A
  pre-shared-key address 192.0.2.2 key SecretKey
!
crypto isakmp policy 1
 encr aes
 authentication pre-share
 group 2
 lifetime 3600
crypto isakmp profile vpn-to-site2-via-provider-a
   keyring vpn-peer
   match identity address 192.0.2.2 255.255.255.255
   keepalive 10 retry 5
!
!
crypto ipsec transform-set vpn-peer esp-aes esp-sha-hmac
!
crypto ipsec profile vpn-to-site2-via-provider-a
 set transform-set vpn-peer

and now it all works. Of course now that it’s working I find a Cisco doc on New Version Site-to-Site Configuration that shows this.

Power & IO Connector for Cisco 2955

If you bought a used Cisco 2955 switch and it didn’t come with the power/IO terminal block connector, it’s a Phoenix Contact 1827761 (Mouser part number 651-1827761).

IPv6 via 6rd on Charter With Cisco IOS

Charter does not yet offer native IPv6 but they do have IPv6 via 6rd tunnels for testing. It mostly works but I’ve seen some weirdness with various v6 destinations. Thumbnail images breaking on Facebook periodically and things like that.

Cisco IOS Configuration

I have this working on a Cisco 881W running IOS 15.3(2)T with the advsecurity license.

Enable IPv6:

ipv6 unicast-routing
ipv6 cef

Set up a DHCPv6 pool:

ipv6 dhcp pool home-pool
 dns-server 2607:F428:1::5353:1
 dns-server 2607:F428:2::5353:1
 domain-name home.example.com

Tunnel interface:

interface Tunnel0
no ip address
no ip redirects
ipv6 enable
 tunnel source FastEthernet4
 tunnel mode ipv6ip 6rd
 tunnel 6rd prefix 2602:100::/32
 tunnel 6rd br 68.114.165.1

Routes & general-prefix:

ipv6 general-prefix DELEGATED_PREFIX 6rd Tunnel0
ipv6 route 2602:100::/32 Tunnel0
ipv6 route ::/0 Tunnel0 2602:100:4472:A501::1

LAN interface:

interface Vlan1 
    ipv6 address DELEGATED_PREFIX ::1/64
    ipv6 nd other-config-flag
    ipv6 dhcp server home-pool

VLAN Configuration in Ciena SAOS

This is a continuation of my notes on Basic Configuration in Ciena SAOS. This page covers configuring VLANs on a Ciena SAOS device

PVID

SAOS defaults to a management VLAN of 127 with DHCP client enabled. The CN3911 has the management VLAN on the upstream ports (9 & 10) while the LE-311v has it enabled on all ports.