16 May 2020

TEI_ASSIGNED

So I’ve had the unlucky experience of having to deal with the PRI connected to a T1 controller card on a Cisco 4321 Integrated Router that fails to come back up “online”, EVERY time the router gets power cycled.

Let me tell you… It is annoying! I don’t even want to try to tell you how many hours have been spent listening to hold music while getting in touch with the provider just to have them “run tests” from their side and say that the layer 1 connection is active, so it’s our problem, not theirs.

Basically, what has been happening is that after the router gets rebooted, the D-channel of the PRI gets locked up and doesn’t want to get in sync with the provider to allow inbound or outbound calls. Whenever we ran “show isdn status”, the layer 2 status would come back as “TEI_ASSIGNED” instead of the desired “MULTIPLE_FRAME_ESTABLISHED”. This PRI is connected to a MGCP gateway and in the end we basically either have to clear counters to reset the interface or rebind and rebuild the MGCP connection before it all comes back up.

Luckily since we have been able replicate the issue on every power cycle (whether or not it was an intentional power cycle), and I kept notes which I’m about to share, we have also been able to repeat the same steps to bring it back “online”.

My disclaimer: This worked for me, in the particular environment I referenced above. I have not tested this in any other environment. YMMV. You copy me at your own risk, and I am not responsible for any changes/damages you cause to your environment. These are purely just my notes for the previously mentioned environment.

From the console run the following command.

4321#show isdn status

Reading the output from the above command, summarized, the desired states are:

  • Layer 1 status: Active
  • Layer 2 status: MULTIPLE_FRAME_ESTABLISHED

If layer 1 is “Active” then it means that the circuit with your provider should be good. If it is not showing as “Active”, then first check that the physical cable is connected and isn’t damaged. If it looks okay, then get on your cell phone and call your provider.

If layer 2 status is “MULTIPLE_FRAME_ESTABLISHED” then you shouldn’t be having problems. Try another test call to verify that everything is working now, then go home. LOL.

However, if you’re in a situation similar to me, then you can’t make any calls and your layer 2 status is showing as “TEI_ASSIGNED”. The first, and easiest, thing to try is to clear the counters. This resets the interface. You can then recheck to see if the layer 2 status has changed. You can use the following command to clear the counters and recheck the ISDN status.

4321#clear counters
4321#show isdn status

If that did not work and your layer 2 status is still “TEI_ASSIGNED” then the next step is to try to restart the T1 controller. Then recheck the ISDN status. Use the following commands to do so.

4321#clear controller t1
4321#conf t
4321(config)#controller t1 0/1/0
4321(config-if)#shutdown
4321(config-if)#no shutdown
4321(config-if)#exit
4321(config)#exit
4321#show isdn status

Did that work for you? Hopefully it did. But if it did not work, then there is one more thing to try. Rebind the layer 3 to call manager and then teardown and rebuild the MGCP connection, and recheck the ISDN. Use the following commands to do so.

4321#conf t
4321(config)#interface serial 0/1/0:23
4321(config-if)#isdn bind-l3 ccm
4321(config-if)#no mgcp
WARNING: no mgcp: Teardown MGCP application may take a while to clean up resources
4321(config-if)#mgcp
4321(config-if)#exit
4321(config)#exit
4321#show isdn status

Okay, now the layer 2 status should be coming back as “MULTIPLE_FRAME_ESTABLISHED” after checking the ISDN status. If it does not, then you really should be beginning to open a TAC case with Cisco to figure out WTF is going on.

Hope this helps someone! Keeping these notes handy has definitely helped me, more than once. Also, FWIW, the one Cisco PDF that I did find somewhat helpful when troubleshooting this was: https://www.cisco.com/c/en/us/support/docs/wan/t1-e1-t3-e3/8131-T1-pri.pdf

Category: Cisco | LEAVE A COMMENT
19 March 2020

CUCM 10.5 – Updating VMtools

Cisco Call Manager is an integral part of any company that runs it for all of their “voice” or telephony services. I’ll be honest… I’m always a little afraid to console in and do stuff on a CUCM server because I don’t feel like I know enough to quickly troubleshoot issues I might cause.

However that doesn’t mean that I can avoid CUCM all together. I do have to jump into a CUCM server occasionally. Like when it’s been virtualized and it’s time to update the version of VMware Tools (VMTools) that is running on it. Thankfully, that task is a lot easier than it might initially seem. I’ll demonstrate how to upgrade the VMTools on a server running CUCM v10.5.2.

In vCenter, select your CUCM server. Dropdown the ‘Actions’ menu and select ‘Guest OS’. Then click on “Install VMware Tools…”.

You’ll see a pop-up message, click ‘Mount’. This will make vCenter mount the VMTools iso in your virtual machine so that the guest OS can access the installer.

Now, login into your CUCM vm’s console as an admin, and enter the following commands.

admin: utils os secure permissive
admin: utils vmtools refresh

You will be prompted that the tools install will reboot the machine twice. Press ‘y’ and hit ‘Enter’ to continue….

If vmtools has not ever been installed on this vm, or if the install didn’t complete, you might see a message that stating that you need to manually restart the server. If so, enter the command it shows to finish the intsall by rebooting the server.

admin: utils system reboot

After the reboots are finished, log back in as admin to your CUCM server. Enter the following command.

admin: utils os secure enforce

That’s it! Your VMtools have been updated. The updated guest OS info should now be visible when you look at this CUCM vm in your vCenter.