21 May 2020

Enabling SSH on Cisco iOS

While telnet and SSH are both allowed types of connections to Cisco gear, there is honestly no reason why you should be using telnet in today’s world. You should be using SSH for accessing all of your network devices. In very simplistic terms [and while the technologies are different], you can almost think of it as telnet being the equivalent to HTTP and SSH being the equivalent to HTTPS.

Telnet transfers all data in clear plain text and thus your passwords or other credentials are visible to anyone watching. Using SSH, means that all of your data is encrypted between the device and your computer, so no one else can see your sensitive bits like passwords. Anything used in production should be secured, and thus SSH is the obvious preference. So lets look at how to enable SSH on our device. Once SSH is enabled we can then disable telnet.

Open a console or telnet session on your device to get started.

The first thing we need to do is make sure that the device is configured with a hostname and a domain name.

CiscoDevice# conf t
CiscoDevice(config)# hostname PWWF
PWWF(config)# ip domain-name it.playswellwithflavors.com

The next step is to allow users that are configured on the switch to login with SSH or Telnet connections.

PWWF(config)# aaa new-model

Next we generate the cryptographic keys that the device will use.

PWWF(config)# crypto key generate rsa

We then want to enable SSH version 2 on the device.

PWWF(config)# ip ssh version 2

We will next set the desired SSH authentication timeout (in seconds). This is the amount of time you have to enter the correct user credentials after connecting. The default value is 120 seconds.

PWWF(config)# ip ssh time-out 60

Then we can change the number of allowed SSH authentication retries that are allowed.

PWWF(config)# ip ssh authentication-retries 3

Next up is to configure all of the line vty (virtual terminal).
We will configure the following :

  • set the input transport to SSH only
  • set the login type to local logins.
  • set the passwords to use strong encryption
  • set a timeout for inactive sessions (in minutes)
PWWF(config)# line vty 0 15
PWWF(config-line)# transport input SSH
PWWF(config-line)# login local
PWWF(config-line)# password 5
PWWF(config-line)# exec-timeout 10
PWWF(config-line)# exit
PWWF(config)# exit
PWWF#

The final step is to save our configuration changes with the following command.

PWWF# write

Now you can close your terminal session and connect to your device over SSH.


You can verify that SSH access is enabled on your device with the following command.

PWWF# sh ip ssh
SSH Enabled - version 1.99
Authentication timeout: 60 secs; Authentication retries: 3

If you have not yet created a user credentials, or if you wish to add a new user, here is the command.

In this example, the user name is “bob” and the password is “Aloha1234”

PWWF# conf t
PWWF(config)# username bob secret Aloha1234
Category: Cisco | LEAVE A COMMENT
16 May 2020

TEI_ASSIGNED

So I’ve had the unlucky experience of having to deal with the PRI connected to a T1 controller card on a Cisco 4321 Integrated Router that fails to come back up “online”, EVERY time the router gets power cycled.

Let me tell you… It is annoying! I don’t even want to try to tell you how many hours have been spent listening to hold music while getting in touch with the provider just to have them “run tests” from their side and say that the layer 1 connection is active, so it’s our problem, not theirs.

Basically, what has been happening is that after the router gets rebooted, the D-channel of the PRI gets locked up and doesn’t want to get in sync with the provider to allow inbound or outbound calls. Whenever we ran “show isdn status”, the layer 2 status would come back as “TEI_ASSIGNED” instead of the desired “MULTIPLE_FRAME_ESTABLISHED”. This PRI is connected to a MGCP gateway and in the end we basically either have to clear counters to reset the interface or rebind and rebuild the MGCP connection before it all comes back up.

Luckily since we have been able replicate the issue on every power cycle (whether or not it was an intentional power cycle), and I kept notes which I’m about to share, we have also been able to repeat the same steps to bring it back “online”.

My disclaimer: This worked for me, in the particular environment I referenced above. I have not tested this in any other environment. YMMV. You copy me at your own risk, and I am not responsible for any changes/damages you cause to your environment. These are purely just my notes for the previously mentioned environment.

From the console run the following command.

4321#show isdn status

Reading the output from the above command, summarized, the desired states are:

  • Layer 1 status: Active
  • Layer 2 status: MULTIPLE_FRAME_ESTABLISHED

If layer 1 is “Active” then it means that the circuit with your provider should be good. If it is not showing as “Active”, then first check that the physical cable is connected and isn’t damaged. If it looks okay, then get on your cell phone and call your provider.

If layer 2 status is “MULTIPLE_FRAME_ESTABLISHED” then you shouldn’t be having problems. Try another test call to verify that everything is working now, then go home. LOL.

However, if you’re in a situation similar to me, then you can’t make any calls and your layer 2 status is showing as “TEI_ASSIGNED”. The first, and easiest, thing to try is to clear the counters. This resets the interface. You can then recheck to see if the layer 2 status has changed. You can use the following command to clear the counters and recheck the ISDN status.

4321#clear counters
4321#show isdn status

If that did not work and your layer 2 status is still “TEI_ASSIGNED” then the next step is to try to restart the T1 controller. Then recheck the ISDN status. Use the following commands to do so.

4321#clear controller t1
4321#conf t
4321(config)#controller t1 0/1/0
4321(config-if)#shutdown
4321(config-if)#no shutdown
4321(config-if)#exit
4321(config)#exit
4321#show isdn status

Did that work for you? Hopefully it did. But if it did not work, then there is one more thing to try. Rebind the layer 3 to call manager and then teardown and rebuild the MGCP connection, and recheck the ISDN. Use the following commands to do so.

4321#conf t
4321(config)#interface serial 0/1/0:23
4321(config-if)#isdn bind-l3 ccm
4321(config-if)#no mgcp
WARNING: no mgcp: Teardown MGCP application may take a while to clean up resources
4321(config-if)#mgcp
4321(config-if)#exit
4321(config)#exit
4321#show isdn status

Okay, now the layer 2 status should be coming back as “MULTIPLE_FRAME_ESTABLISHED” after checking the ISDN status. If it does not, then you really should be beginning to open a TAC case with Cisco to figure out WTF is going on.

Hope this helps someone! Keeping these notes handy has definitely helped me, more than once. Also, FWIW, the one Cisco PDF that I did find somewhat helpful when troubleshooting this was: https://www.cisco.com/c/en/us/support/docs/wan/t1-e1-t3-e3/8131-T1-pri.pdf

Category: Cisco | LEAVE A COMMENT
19 March 2020

CUCM 10.5 – Updating VMtools

Cisco Call Manager is an integral part of any company that runs it for all of their “voice” or telephony services. I’ll be honest… I’m always a little afraid to console in and do stuff on a CUCM server because I don’t feel like I know enough to quickly troubleshoot issues I might cause.

However that doesn’t mean that I can avoid CUCM all together. I do have to jump into a CUCM server occasionally. Like when it’s been virtualized and it’s time to update the version of VMware Tools (VMTools) that is running on it. Thankfully, that task is a lot easier than it might initially seem. I’ll demonstrate how to upgrade the VMTools on a server running CUCM v10.5.2.

In vCenter, select your CUCM server. Dropdown the ‘Actions’ menu and select ‘Guest OS’. Then click on “Install VMware Tools…”.

You’ll see a pop-up message, click ‘Mount’. This will make vCenter mount the VMTools iso in your virtual machine so that the guest OS can access the installer.

Now, login into your CUCM vm’s console as an admin, and enter the following commands.

admin: utils os secure permissive
admin: utils vmtools refresh

You will be prompted that the tools install will reboot the machine twice. Press ‘y’ and hit ‘Enter’ to continue….

If vmtools has not ever been installed on this vm, or if the install didn’t complete, you might see a message that stating that you need to manually restart the server. If so, enter the command it shows to finish the intsall by rebooting the server.

admin: utils system reboot

After the reboots are finished, log back in as admin to your CUCM server. Enter the following command.

admin: utils os secure enforce

That’s it! Your VMtools have been updated. The updated guest OS info should now be visible when you look at this CUCM vm in your vCenter.

20 December 2019

File management – Cisco Flash

When working on Cisco devices, sometimes you have to delete files from the device. It’s moreless the same on all Cisco devices. While there are other file systems available to you on your device (see first command listed), Cisco devices use “flash:” as their default file system. My examples will also use the default “flash:” but note that the commands will work the same on any other available file system, such as “usb0:“.

Show Available File Systems
Device# show file systems

Show Files
Device# dir flash:

Create Directory
Device# mkdir folder

Step into Directory
Device# cd folder

Delete File
Device# delete /force flash:/filename
Device# delete flash:/filename

Delete Directory
Device# delete /force /recursive flash:/folder

Copy File
Device# copy usb0:[/directory]/filename flash0:[/directory]/filename

Category: Cisco | LEAVE A COMMENT
18 December 2019

Cisco 3850 – Upgrading IOS

Upgrading the IOS version on your Cisco 3850 Catalyst switch is not something to be dreaded. In fact it should probably be rejoiced, as these days it most likely means you’re also catching up on patching some vulnerabilities on your device. While the upgrade process is a little different on the 3850 then older generations of the Catalyst switches, not much has changed. Overall it is an easy process that won’t take you long to complete.

What has changed is that the 3850 switch? Well the “operating system” that the hardware now runs on is called IOS-XE. Whereas previous generations of Cisco hardware ran on IOS software.

The IOS-XE software is delivered as a bundle containing a set of packages which your device will use. These packages, during the install step, will get expanded before they are actually installed on the device. The older generations of devices that ran IOS had a .bin file that was essenitally a “flat” image and not a bundle of packages. While there are other changes too, that is the main difference between the “new” IOS-XE and the “old” IOS.

Okay… Lets get our switch upgraded…

Step 1: Download the latest (or rather the recommended) IOS-XE version from the support portal of Cisco.com. (At the time of me writing this, the recommended release version is “Fuji-16.9.4”)

Step 2: Transfer the .bin file you downloaded in step 1 to your device. There are a few ways to so; TFTP or USB being the most common.

  • Via TFTP

3850-SW# copy tftp flash:
Address or name or remote host []? 10.1.2.3
Source filename[]? cat3k_caa-universalk9.16.09.04.SPA.bin
Destination filename[cat3k_caa-universalk9.16.09.04.SPA.bin]?
Accessing tftp://10.1.2.3/cat3k_caa-universalk9.16.09.04.SPA.bin...
Loading cat3k_caa-universalk9.16.09.04.SPA.bin from 10.1.2.3: !!!!!!!!!!!!!

  • Via USB

3850-SW# copy usbflash0:cat3k_caa-universalk9.16.09.04.SPA.bin flash:

Step 3: Start the Install.

3850-SW# software install file flash:cat3k_caa-universalk9.16.09.04.SPA.bin

Once the software install completes, you will need to reboot the switch so that it can boot into the freshly installed IOS-XE version.

3850-SW# reload

Step 4: Verify the new version. After rebooting the device in the previous step, you will want to verify that you are indeed running the new IOS version. You can run the following command to do so.

3850-SW# show version | i Switch
Cisco IOS Software [Fuji], Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 16.9.4, RELEASE SOFTWARE (fc2)
Switch Ports Model SW Version SW Image Mode

Here we can see that the switch is running Fuji, and that it is on version 16.9.4. And now we’re done!

Category: Cisco | LEAVE A COMMENT