200 OK BroadWorks Data Center Moves Made Easy (as possible)


Many BroadWorks service providers are migrating to new data centers and cloud hosting providers. This change brings many aspects into consideration, such as

  • Rollback recovery options
  • Managing BroadWorks Configurations
  • Choosing Operating System versions

To look more into the migration, Mark Lindsey spoke with ECG AMTS Ashley Lee for a Q&A. Ashley is an ECG Associate Member of Technical Staff and Software Support Engineer, he provides technical expertise and creative solutions for the enterprise and service provider markets. His clients include CenturyLink, Momentum Telecom, UVA, and Vonage.

Mark: Will the existing servers be migrated or will new servers be setup to replace the original BroadWorks Servers? 

Ashley: When conducting a migration, you may want to upgrade to newer hardware or use different hardware due to the distance between the original and new data center.  Using new hardware also provides you with a rollback option should, as the original hardware will still be in place in the original data center during the final switchover/migration.

Mark: Will the BroadWorks servers require new IP addresses and hostnames in the new data center? 

Ashley: In some cases using the same IP addresses will not be available in the new data center.  This is generally the case when your BroadWorks platform is using private addresses. Additionally, if your original systems are using hostnames which identify the location, then you may want to update the hostnames to either remove the location identifier or update the identifier.  Changing either of these, the hostname or IP, require additional BroadWorks configuration adjustments. Several of these adjustments are outlined in the BroadWorks Documentation. However, there are also other items you will need to ensure are updated, such as internal DNS, /etc/hosts files on all servers, and some BroadWorks settings/references not mentioned in the BroadWorks documentation/guides.

Mark and Ashley on the front porch of Valdosta, Georgia's historic 1898 Crescent

Mark: How do you decide which OS to use when migrating BroadWorks servers?

Ashley: When choosing which supported OS to install on the new BroadWorks servers there are a few things to consider.  First, what OS is installed on the original server? Second, is this OS still supported? Finally, what are the impacts of running 2 different OS versions between the peer BroadWorks Servers? 

Generally it is recommended to install the same OS on the new BroadWorks servers, primarily due to the ease of maintenance and updates for the servers.  However, if the OS installed on the original BroadWorks server is no longer supported, it is recommended to upgrade to a supported OS from a security standpoint.  Installing a newer/supported OS will also mean you will not have to upgrade this server’s OS when the peer server is scheduled to be upgraded later. This will save you time and help out with planning the upgrade of the peer server. 

Lastly, you may ask is it supported to run 2 different OS versions between peer servers and more importantly what impacts does this have on the BroadWorks servers, especially the AS and DBS.  To quote BroadSoft TAC, “BroadWorks for the most part is OS blind.” This means generally there will not be any impact between the BroadWorks Servers. Additionally in ECG’s experience we’ve seen a production BroadWorks platform upwards of 30,000 subscribers, run successfully for 6 months with different OS versions installed on each geo-redundant peer, this includes the AS and DBS.  At the end of the 6 months the peer server was upgraded to a matching OS. The only consideration here would be if any of the 3rd party applications currently in use are not supported by the new OS. In this case it would be recommended to upgrade the 3rd party application to a supported version for the new BroadWorks server’s OS. 

The final caveat here would be the DBS.  In some older OS versions, the DBS requires additional 3rd party software or drivers, which must be manually installed to match the current kernel version.  In newer OS versions, this software is available via repositories and is much simpler to install and manage.

Mark: What are the special challenges in relocating a BroadWorks Database Server (DBS)? 

Ashley: There are a couple of special challenges with relocating a BroadWorks DBS.  To be specific, if you are simply relocating the BroadWorks servers and not changing the IP address or hostname, then the move should be fairly simple and straightforward.  However, if you are changing the IP address/hostname of the DBS, then you will need to plan accordingly as the procedure to adjust the IP address/hostname of the DBS vastly differs from other BroadWorks Servers.  This is in part due to the scripts which BroadWorks Documents outline, but also due to the database itself referencing the hostname in several places.

In some cases it would be easier to simply reinstall BroadWorks DBS on the server and set the server up as a new peer.  This would likely be the easier option as with ECG’s experience, not many engineers are familiar with the scripts recommended by the BroadSoft documentation.  Also should something fail during the execution of one of the scripts, it is generally faster to reinstall BroadWorks DBS software, than to attempt to recover or fix what has been mangled by the script.

Mark: In the past, you've mentioned a special challenge in planning disk space for the BroadWorks DBS. Can you tell more about that.

Ashley: Generally when you are installing a new BroadWorks server you will need to ensure you follow the BroadWorks documentation and guides.  However, in a migration you already have a server which is healthy and working fine. In most cases you can audit the existing server to confirm how the new server should be built.  For most of the BroadWorks servers, auditing the existing server should be fairly easy and hard to miss any fine details. For example on an AS or NS, executing ‘df -h’, will generally provide you with the drive setup for the new server.  However, for the DBS, ‘df -h’ will not provide you with information regarding the drives setup for the Oracle database.  If your audit is not conducted by someone who is knowledgeable in regards to BroadWorks DBS installation and setup, then the additional drives for the Oracle database may be neglected.

In the event, the DBS’s new chassis has room for additional drives, omitting these drives from the audit can be remedied easily.  However, should the chassis not have room for the additional drives, then the hardware would not be usable. In some cases you may expect to be able to use a NAS to resolve the problem with missing drives.  However, using a NAS for the Oracle drives is not recommended. This is due to the DBS requiring access to the NAS at all times, and by using a NAS, your DBS in the new Data Center is dependent on two different devices always being online.