Voicemail redundancy and failover

[Neiby]

I'm getting confusing answers about this from our local reseller, so I thought I'd ask this question here. We are concerned about disaster recovery and failover, and I'd like to find out how to design a redundant voicemail system with ShoreTel. I was told that every user has a mailbox on Director, but that it was possible to have distributed voicemail servers.

As it was explained to me, the users would get their voicemail from the distributed voicemail servers. But what happens if Director is down for some reason? Does that mean that no new voicemails can be left even though the DVMs are still available?

[VonZipper]

You can build a replica of your server and use Doubletake software. If the main server fails, the second server takes over with no loss.

[Neiby]

That would work if the server were at the same location. I want a backup mail server at a different location. As I understand it, it doesn't look like the distributed voicemail servers buy you anything. They don't save you any bandwidth because the voicemail still has to go across the network from Director to the DVM, and the DVM can't take new voicemails if Director is unavailable. I guess I don't see the point in having them.

It looks like they'd give you the ability to listen to old voicemails while Director was down, but I'm more concerned about the ability to leave new voicemails during a DR situation.

[VonZipper]

We do not use DVM's at this point. We have 9 locations (500 users) in Washington and Oregon with a central server in Washington. We have not failed over to the backup server once yet in 5 years though.

[Neiby]

DR is a big deal for our management. We have to consider and plan for DR situations. Voicemail redundancy and failover is important, as is redundancy for Director.

[VonZipper]

Well, just a second....why couldn't the back up server be at another site? Well, I suppose that would be a little scary though. Doubletake watches for a heartbeat signal from the main server. If that ever goes away, it takes over at once. It would be spooky if the network connection between the two went down, because the other server would start tryin gto talk on the network, but if it is a reliable connection, I would not worry about it to much.

We worry about redundancy too, but we have built our netwrok so that it would take alot to take us off the map.

What are you doing for redundancy for your application servers?

[Forum]

My favorite quote.... Shoretel provides Resiliancy, But budget determines Redundancy.

I have a feeling you do not have a clear picture of how DVM works, and it really hard to explain over the internet.

I will tell you this though, there are other ways to deal with DR. My favorite is Symantec LiveState with BareMetal restore. It makes a contiuous backup (disk to disk over the network would be awesome) and allows you to restore to any machine, including VMWare. So...You have an offsite copy of this backup, the building burns down, you just power up your laptop with VMWare and restore to your virtual server. VMWare is not technically supported by Shoretel, but a few people are using it.

[Neiby]

Well, just a second....why couldn't the back up server be at another site? Well, I suppose that would be a little scary though. Doubletake watches for a heartbeat signal from the main server. If that ever goes away, it takes over at once. It would be spooky if the network connection between the two went down, because the other server would start tryin gto talk on the network, but if it is a reliable connection, I would not worry about it to much.

We worry about redundancy too, but we have built our netwrok so that it would take alot to take us off the map.

What are you doing for redundancy for your application servers?

We have secondaries for our critical services at a DR site, which is where I'd like to put a backup Director and voicemail server.

Charles is right, I don't understand how DVM works. My understanding is that Director does all the heavy lifting for voicemail, but it can push out voicemails to a DVM so that users can hit the DVM to listen to voicemail instead of going back to the Director server. The DVM servers do not directly accept incoming voicemails from callers, they just playback recorded voicemails pushed to them by Director. Is that correct?

[cburgy]

The role of the DVM is a bit confusing and it wasn't quite what we expected when we implemented our first few. That being said they are responsible for voicemail at the remote site. If you hit voicemail at a remote site you'll notice that it goes to the HQ vm extension first but then redirects to the DVM. Voicemail is taken at the DVM server and when a WAN connection is severed voice mails and auto attendants are still served by the DVM so as long as you don't restart the DVM. If you have a site power failure and the WAN link is down, the DVM won't come up and be able to respond to requests. The PCM communication is also handled by DVM but only have the client first speaks to HQ and is redirected to the DVM at their site.

I know they are working on database sync across the DVMs and distributed director. Once they have that accomplished, I think DVMs will function the way we think they should.

[Neiby]

That doesn't sound like a very resilient solution, but it sounds like it will be much better once they resolve this database sync issue. It's probably unreasonable to design a redundant VM solution based on the assumption that power will be uninterrupted in a DR scenario.

[devdevil85]

Does ShoreTel provide a whitepaper or some form of documentation for DVM's? I have only found information on other websites (Dr. VoIP) and this forum.

[Palitto Consulting]

You should be able to find sufficient information in the documentation PDF's (Administration Guide and Planning and Installation Guide) accessible from the Director menu.

[400degreez]

The issue is the database location. As far as back ups, just back of the folders. across the wan. If your company needs that much DR you would already have it, right.

[southeasternMI]

The DVM isn't a true backup or failover. The key to the DVM is to assign the remote users, VM to the DVM which will reduce traffic on the WAN and when you add AA and workgroups to the the mix you really reduce traffic. If you lose connectivity between sites both sites can continue with buisness as long as both sites have local trunks. Its not the best way to have failover/ backup but its helps.

[sschnei]

Here is a pretty good write up of how redundancy is built into the ShoreTel system

At least one server is always required in a ShoreTel deployment and is referred to as the Headquarters (HQ), or root, server.

Additional servers can be added for better fault tolerance, redundancy and survivability. Whether referred to as Distributed Voice Mail servers (DVMs) or, more recently, Remote Application Servers, these additional servers rely on the HQ server to receive configuration changes but, otherwise, can run entirely independent from the HQ server for extended periods of time.

An additional server can be added to a remote site to keep local Auto Attendant (AA) and Voice Mail (VM) traffic off the WAN and to provide survivable AA & VM access even when connectivity to the Headquarters site/server has been lost.

An additional server can be added to the Headquarters site to enable failover & redundancy in case the HQ server is taken off-line for maintenance or upgrades.

All the servers communicate with each other and work together to "back each other up" in case one server is unreachable.
ShoreGear Switches: Selection of Primary and Secondary Servers
Each ShoreGear switch, starting with full knowledge of all ShoreTel servers in it's own site and all sites above it, will select a primary server and a backup, or secondary, server. When a ShoreGear switch needs any server-based service (such as an Auto Attendant prompt or a Voice Mail box) the switch will first try to connect to its primary server and, if that server is unreachable, will connect to it's secondary server.

The selection of primary and secondary servers is done dynamically by the switches and is done by each switch individually and from its unique position and perspective of the ShoreTel site hierarchical tree. If there are two (or more) ShoreTel servers at a site, then the ShoreGear switches at that site will always select those local servers for their primary and secondary servers. In fact, different switches will select different servers for their primary and secondary to effectively load-balance across all servers at a site.

If there is only one server at a site, then that server will always be selected as the primary server for each switch at that site. The switches then select the next "nearest" server by looking up the ShoreTel site hierarchical tree. This hunting will continue up the tree until each switch has selected both its primary and secondary server or it reaches the HQ site and has run out of servers to choose from. In practice, with most simple, hierarchical tree structures, this often will mean that each switch will use it's local DVM server as it's primary server and the HQ server as its secondary server.

Note: When a switch is selecting it's primary and secondary servers it will always hunt up the site tree (towards parent sites and the root/HQ site) and will never hunt to the side or down the tree (to a sibling site or a child site).


Distributed Server Services
Many of the ShoreTel server-based services are fully distributed, such as all AA prompts and the recorded names of all users. When one of these services is needed by a ShoreGear switch it will connect, as always, to its primary server and that server will be able to service the request locally.

But some server-based services are performed only by one, single server, such as the storage of each individual user’s Voice Mail messages and greetings. When a ShoreGear switch needs, for example, to route a caller to leave a voice mail message for a particular user the switch will connect to it's primary server and that server will then redirect the call to the appropriate server that hosts the specified user's voice mail box.

Yet still other server-based services are performed only by the Headquarters server and are not distributed at all (At least not yet! There's always room for improvement!)

Some services are assigned to a specific server and, though distributed, are not redundant. This allows the ShoreTel system to scale to a large number of sites, switches and users and to provide remote site survivability but doesn't necessarily provide redundancy. Our distributed Telephony Management Services (DTMS) is a great example of this. TMS controls the management of a set of ShoreGear switches and the users that are serviced by those switches, including many of the desktop call control functions of Personal Call Manager. This means that a remote site with it's own server can be fully independent and functional even during loss of connectivity with the HQ site/server. But if that remote site's DVM goes off-line the Call Manager control for the users at that site does not fail over to another server. For these services, growth, scalability and survivability have been achieved but not redundancy.

The functions that each server performs depends on whether it is the Headquarters server or if it is one of the additional/remote/distributed server(s).

Since Servers fail from "child" to "parent" you may consider the HQ server at the DR site and DVM's as your main voice mail servers. The Double Take program is now suppose to be able to replicate to different IP sunbnets, but I am unable to find if it is supported across a WAN environment.