Jay Taylor's notes
back to listing indexUnicorn Switch III: Brocade ICX6450 - Review and Impressions : homelab
[web search]Unicorn Switch III: Brocade ICX6450 - Review and Impressions
This is a follow-up to this post I made earlier this week; it has a links to my first and second attempts at finding a Unicorn Switch.
I received my Brocade ICX6450-24P on Wednesday. Unfortunately, for me, I had a maintenance at work Wednesday night and didn't even make it home until about 1:30 in the morning. So, I only had about a half an hour to mess with it, because I had to be back to work by 10am and I have an hour commute. I did spend some time messing with it last night though.
For starters, it's... not LOUD, but annoying. there's a bit of a whining pitch to the fans. It's one of those things where you can easily have it running in the same room and ignore it without too much trouble, but when the sound cuts off it's still a bit of a relief. Definitely quieter than an R710, but I might still look at swapping fans. Really shouldn't be a big deal in the basement where my rack is though.
Idle power at the wall comes out at 37 watts, though at first power up and while booting (and interestingly, while the fans are running at max), it's coming out at 25w. I suspect this is because the PoE circuitry isn't active at that stage and that's what's taking up the extra power. Given that my 3750E is idling at 105w (on the switch, not at the wall) with 12w of PoE devices, I'll take cutting my switch power bill in half or more quite happily.
Now, we use Brocade at work, as I mentioned in my last post, so I knew some of what I'm about to pass on, but it's much more irritating when you're doing it by hand at home instead of with a pre-configured script. Brocade has, in my opinion, some of the dumbest configuration options ever. For example, if you want to set a default description on your ports on a Cisco (say, to just set it to --DISABLED--), you would do interface range gi1/0/1-24
to get all the interfaces, followed by description --DISABLED--
. Brocade will not allow you to apply a port-name (their version of description) to more than one port at a time. But this is a minor issue, that up-arrow can help with greatly.
No, where you get pain is the VLAN and inter-VLAN routing configurations. Because you cannot go into a port, set what VLANs it's a member of and whether those VLANs are tagged or not, then move to the next port. You can also not look at the port configuration to see what VLANs are applied to just that port. No, all of this is done through the VLAN itself. So in configuration mode, you type in vlan 100
, which puts you in a mode to configure that VLAN. Untagged ports are untagged e1/1/1 to 1/1/5
to add your first five ports as access ports to VLAN 100. Substitute tagged
for a Cisco-style trunk port. Want another VLAN on there? Switch VLANs and add it. Compare this to Cisco's interface gi1/0/1
followed by switchport mode trunk
and you see why it's irritating. Even if you only want a few VLANs to cross it, Cisco is switchport trunk allowed vlan 100,110,200-250
for example.
Want your "trunk" port to have a native, untagged VLAN? This requires issuing dual-mode <vlan-number>
on the interface itself. But, if that interface isn't a TAGGED member of the VLAN already, this command will fail.
For inter-VLAN routing, you need to define a virtual-ethernet interface in each VLAN. This isn't much of a big deal, really. All you do is, when you're in the VLAN, issue the command with the interface number (I use the VLAN number for obvious reasons) like thus: router-interface ve100
. So let's go create a new VLAN. When I was doing this I wanted to create all the VLANs and get everything set up for inter-VLAN routing before figuring out which ports I wanted to plug in where.
conf t
vlan 110
router-interface ve110
All of that completes with no issues, and is the correct way to configure VLAN 110 for Inter-VLAN routing. However, if you try to go at this point and configure an IP address on interface ve110, the switch will tell you it doesn't exist. If you go back into VLAN 110 and re-issue the router-interface command, it will tell you it already exists. Further, if you do a show vlan
, VLAN 110 does not show up there. This is not a bug folks, it's by design. Neither the VLAN nor the ve110 interface will show up (or be accessible, in the ve110 case) until there is at least one port assigned to the VLAN.
Once there is, everything works pretty much as you would expect, so if you're pre-configuring your switch, you really want to know what port you're going to use for trunking; at that point you simply add tagged e1/1/1
(or whatever) to the list of commands when configuring the VLAN.
There's other issues too, though some of them are trivial; for instance when you do show interface brief
, which is their version of Cisco's show interface status
command, the description field to the right only holds 8 characters. You can expand that to 15 characters with show interface brief wide
, and nobody I've talked to at Brocade/Ruckus seems to know why that isn't a default for the command. For the most part though, if you know Cisco and you use your tab-complete and ? keys, you can figure out Brocades well enough.
Support is... interesting. Note I mentioned Ruckus above; Brocade has been bought up by Broadcom specifically for their fibre-channel business, and have sold their datacenter IP networking stuff to Extreme. The Ruckus Wireless and ICX/FCX/FastIron product lines were sold to Arris (the cablemodem maker) and is being rebranded under the Ruckus name. When I went to Ruckus to look for software updates, I found them, and I was able to download both the current and the recommended software releases with a simple click to agree to the (surprisingly short) EULA on their website.
What I couldn't download was their release notes for any of that software, nor could I access KnowledgeBase articles on basic things like licensing. So you can get the firmware but you may be flying blind. I recommend sticking with the recommended software release if you want to upgrade.
Speaking of licensing, there are two types of license for this switch; the Premium License (enables OSPF, RIP, and assorted other Layer 3 goodies that aren't generally needed in a homelab core switch), and PoD, or Ports on Demand license. This is the lovely practice of disabling uplink or stacking ports on the switch in order to "make it cheaper" to buy the switch; you then pay through the nose later to re-enable these ports. For the 6450, that amounts to two of the four SFP+ 10G ports being disabled without the license.
Brocade used to offer an evaluation license to activate the ports, and I don't know if they were right-to-use (in other words, the ports stayed active afterwards if they were in use) or not, but I couldn't find any on Ruckus' website. This may simply be because I'm using my personal email without an ICX 6450 registered with Ruckus, but it may be they don't offer them either.
Overall, I'm pleased enough with it. The price-point on this one is clearly in the realm of "good deal because people look at paint spatter and think it's dirty and damaged", so in that one regard it doesn't quite meet all the requirements for a Unicorn, but it comes closer than anything else. If you're going to look for one as a deal, you probably can't go far wrong with Juniper EX2300s/3300s, or Brocade ICX6450s/6610s. What you really should look at is what you might be willing to sacrifice if you're not willing to wait.
If IPv6 isn't necessary for you, Dell PowerConnect 55xx will work fairly well, and has HDMI stacking. (Also good if you have aversions to CLI for some reason; the PowerConnect Web UI isn't terrible.)
If power isn't a problem and you're willing to use fiber instead of DACs, Cisco 3560E is a good choice. Go to a 3750E if you might want to expand 10G ports; 3750Es stack and a pair of 24-port 3750Es gives you 48 gig and 4 10G.
Another one that works for lack of IPv6 is the Aruba S2500 Mobility Access Switch. Has integration with their 7k series controllers if you're inclined to play with that, and has the advantage of 4x SFP+ ports. Consumes a tad more power, but also is unrestricted by licensing and can do OSPF. If you're looking to learn enterprise wireless at all, this is probably an excellent choice.
I bought a 6430 second hand in the summer, thanks for pointing out that the firmware was available, but damn, its weird that the Release Notes are behind a contract when the firmware isn't.
Yeah, it's REALLY weird. That's why I'm sticking with 08.0.30h (the recommended) instead of pushing my luck with the latest.
How's the 2 Month review coming? Everything been working so far?
Yep, haven't had any real issues. I did have a secondary switch in my office that was linked up at 100 meg - which was mildly disturbing as to why all of a sudden I couldn't get over 10MBytes/sec - when on the 3750E it had been connected at gigabit, but replacing the jumper from the wall jack to the switch fixed that. Basically, Brocades are a bit more finicky about crappy cables than Ciscos are.
One of my complaints about the VLAN handling though, that's gotten better. You can go into global config mode, type vlan 100, 150, 200-250
and it will let you apply a group of ports to VLAN 100, 150, 200, 201, 202, etc. all at the same time. There's a max limit on the number of VLANs at one time - I think it's 64, but I don't recall - and it will not ignore previously non-existent VLANs (because you could be trying to group-create them), so you can't do vlan 1-1000
and only get the ones that existed - it will create all the others (and inevitably choke because there's too many VLANs in the range).
Overall it's been solid; I just wish I could find a Ports-on-Demand license for the thing that doesn't cost hundreds of dollars. At this point it'll be cheaper to get another one and stack it, or get a LB6M and do LACP.
I am intrigued and fascinated. I would like to subscribe to your newsletter. (PM incoming)