Brents IT Blog

Random thoughts by an IT GOAT


Brighthouse Fiber Services Review

About 4 year ago we started to look into network issues we kept having with Paetec(now Windstream) MPLS services.  A few outages had occured with more frequency than normal and the question came up, "what can we do better".  The answer, as it turned out, was to switch from copper based MPLS circuits to Fiber Metro-E.

A few considerations had to take place before a final decision could be made, so lets dive into this:  1. We needed a provider that could reach all of our locations.  2. We needed a provider would would take responsiblity for their actions and uptime.  3.  The provider must be able to provide services that meet our current demands but also be able to easily transition to higher demands.  4. Finally, the provider must do 1,2,3 while staying in the most cost effective price range.

Brighthouse Fiber Business services met all the of the requirements with the exception of number 1.  They were however able to provide services to all but one of our facilities, which the one facility we had an issue with had a working fiber network near it that they could tie into from one of our existing sites.  They were also able to complete build outs in a timely manner and eat most of the connection costs as long as we signed a 3 year contract(which was acceptable to our company needs). 

So why Fiber Metro-E?  Fiber connections are newer connections, they are not subject to lightening issues, power can be pushed from the main location, the response time is generally below 10ms(we average 3-5ms), connection speeds can be upgraded over night up to 1Gb speeds(this now take longer because of stability planning, but they can rush order if necessary), the cost for the connection is much lower than an equal copper/digital line, the install time is usually less than 60 days(expansion of services is much less), and finally they can provide Metro-E/Internet/Voice/Video at the same time.

If you are looking at the two, consider this:  Is the copper in your area for MPLS expensive?  Is the copper new or very old and suseptiable to water intrusion(bad pair) outages?  Do you have frequent thunderstorms that damage your equipment?  Do your existing MPLS circuits randomly cut out with no reason for 5 minutes at a time? 

Review (So back to our experiences.... )

Data/Internet:  Over the past few years, our uptime has averaged 99.99% uptime.  I should mention that the first year we were on it, they did have some growing pains with us, but after about 18 months, outages dropped to 0.  They do most of their maintenance between 2-4am and they notify us about 2 weeks in advance.  If you have a problem with one of the maintenance windows, just call them up or forward it to your account manager and they will work with you, they are very sensitive to our needs when it comes to preventable outages.

Phones:  I have found their phone service to be quite solid.   I wish i could say its more solid than B1 lines, but as with different technologies, they each have their problems.  We had to work extensively with our hardware configuration to get our VOIP phone system working properly with their SIP connections.  We actually convert their SIP to PRIs using an Audiocodes device they provided.  This device is usually the issue when a problem arises, so much so that recently they replaced our Audiocodes unit with another one because it couldnt correctly handle 4 PRIs.  The change out was quick and we havn't had an issue since.  So take your time switching over, dot your I's and cross your T's.  I should also note that if you plan to centralize your phone system like we did, make sure to tell them you will need 911 numbers associated with different addresses.  I could probably take credit for getting them to work the issue out quickly, but then again, i am sure we were not the only one with this concern.

We recently added a SIP trunk to a secondary phone system we use for outbound appointment reminders.  That installation went off without a hitch, even the installation was quick.  We havent had any issues with the SIP trunk, probably because it connects directly to the PBX server instead of going through converters.  The great part is the phone system used to do this is actually a free one called "PBX in a flash" which uses astrisk and FreePBX to provide a solid VOIP platform.  We then had the PBX in a flash founder alter his teleyapper script in order to allow us to automate appointment reminders to patients.  We process about 350 calls in about 45 minutes using the 24 channel SIP trunk.

Transition:  We transitioned to Fiber at all 9 locations within a 6 month period.  They could of done it faster for us but with our limited IT staff, we did not want to push our luck.  Transitions were seemless at every location, infact it took more effort on our part than theirs when it came time to swap connections.

Conclusion:  Its a great solid product that i would recommend any business that has access to it, switch to.  The price is usually less than a telco can offer for the speeds provided, the connection is solid and you will never experience regret making this change over.  So if you want to lower your blood pressure and concentrate on other issues, Brighthouse Fiber services needs to be at the top of your list.    

Update( 7/24/12 ): On the 24th of July we had an outage that lasted from 5am to 7pm.  The outage was caused by a firmware upgrade to our Brighthouse provided router.  Apparently they have never had an issue with any of them but we became the first exception.  The outage lasted all day and it was mainly due to the fact that overnight technician did not hand off the issue correctly to the day technicians tasked with providing a fix.  The day technician did not bring the required gear to fix the onsite unit in case of a unit failure and then when they went to retrieve a replacement unit, a similar unit was not available and the entire team had to jump on and configure a different unit to work like our previous unit(basically the could not drag and drop the old configuration, they had to create a new one).  With our primary internet, WAN and phone lines all going through this one unit, the failure had significant impact on our day operations.  We did have backups to some of our sites, but without phones at most locations, many back office staff were unable to complete daily tasks.    

The good news is that Brighthouse did conduct a full assessment of what occurred and are making adjustments to way their staff handle upgrades and when they perform them.  They are also working to ensure the proper technology is provided to field technicians and stocked on trucks.  It seems our outage provided a wakeup call to a flaw in their network recovery process. 

So while I don’t like being a guinea pig, I cant say I have ever had an ISP with 100% uptime.  Granted 90% of this outage could of been avoided, but it also provided insight into our own shortcomings and allowed me to present additional solutions to help minimize the damage when an outage of this scale occurs again.