10 April 2022

Upgrade CentOS 8 to CentOS 8 Stream

With CentOS 8 now EOL, it is officially time to upgrade CentOS 8 virtual machines to CentOS 8 Stream. The good news is that it is even quicker and easier than the upgrade from CentOS 7 to CentOS 8 was.

First things first… Take a backup of your virtual machine, or at least a snapshot so that you have something you can revert back to if something goes wrong in this process.

Take a look at what release your CentOS machine is currently running. 

cat /etc/centos-release
cat /etc/os-release

As you can see this machine is currently on CentOS 8.5.2111.

CentOS release version info

At this point, I’m going to enter “sudo su” on my VM and then enter my credentials, so that I can continue as ‘root’ and I don’t have to type “sudo” before every single command.

To begin, start by updating your system.

dnf -y update

The next step is to update your machine to the current CentOS Stream release package. 

dnf -y install centos-release-stream --allowerasing

This step repoints the machine to the CentOS Stream repository rather than the CentOS 8 repository.

sudo dnf swap centos-linux-repos centos-stream-repos

List and view all of the enabled repositories. You should see they are set to “CentOS Stream 8”.

sudo dnf repolist
updated CentOS repo list

Next, synchronize all of the installed packages on your machine.

Note: For situational awareness, this step will upgrade or downgrade packages to match the new CentOS Stream ABI/API and will apparently break 100% RHEL compatibility due to the ABI/API change. This is the perfect example of why you would want to take a full backup of the system before making any changes, just in case the ABI/API change breaks one of your applications running on the system. 

dnf -y distro-sync

Reboot your system.

init 6

Confirm that we are now running on CentOS 8 Stream.

cat /etc/centos-release
cat /etc/os-release

We can now see that this machine is now running on CentOS Stream 8.

Confirmed updated CentOS 8 Stream
Category: CentOS, Linux | LEAVE A COMMENT
9 April 2022

Upgrade CentOS 7 to CentOS 8

Warning: CentOS 8 has reached End of Life (EOL) and is no longer supported. You should really consider moving to a supported OS such as CentOS 8 Stream.

I was looking at some virtual machines earlier today and I realized that they were not running the most current version of CentOS. Since I am going to upgrade them, I figured it’d be the perfect time to document the process of how to do it.

The first thing I do is make a backup of my virtual machine. You can’t recover from an accident if you don’t have a recovery point. At the very least, make sure you have taken a snapshot of your virtual machine.

Next, I verify what version of CentOS I’m on by running the following command. 

cat /etc/centos-release

From the screenshot below you can see that I am currently on version 7.9.2009.

Check CentOS version

At this point, I’m going to enter “sudo su” on my VM and then enter my credentials, so that I can continue as ‘root’ and I don’t have to type “sudo” before every single command.

First step is to install the EPEL repository.

yum -y install epel-release

Next, install both ‘yum-utils’ and ‘rpmconf’ by using this command.

yum -y install yum-utils rpmconf

Next, use ‘rpmconf’ to resolve the RPM packages that are in use on your VM.

rpmconf -a

Then clean up any packages that are not required by your system.

package-cleanup --leaves

package-cleanup --orphans

Go ahead and reboot the system.

init 6

Log back in and do “sudo su” again.
CentOS uses the dnf package manager as its new default package manager, so time to install it.

yum -y install dnf

With dnf installed, it is time to remove the yum package manager.

dnf -y remove yum yum-metadata-parser
rm -Rf /etc/yum

Update all of the dnf packages.

dnf -y update

The next step is to install the CentOS 8 release package.

dnf -y install http://vault.centos.org/8.5.2111/BaseOS/x86_64/os/Packages/{centos-linux-repos-8-3.el8.noarch.rpm,centos-linux-release-8.5-1.2111.el8.noarch.rpm,centos-gpg-keys-8-3.el8.noarch.rpm}

Then upgrade the EPEL repository.

dnf -y upgrade https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
rpm --import http://download.fedoraproject.org/pub/eprl/RPM-GPG-KEY-EPEL-8

Next, clean up the dnf cached files.

dnf clean all
rm -rf /var/cache/dnf

CentOS Linux 8 had actually reached the End Of Life (EOL) as of December 31st, 2021. Which means that CentOS 8 will no longer receive development from the official CentOS project. After that EOL date, if you need to update your CentOS (yes, that means us right now), you need to change the mirrors to point to vault.centos.org where they are archived. So a better option would actually be to upgrade to CentOS Stream instead, but we’ll save that for another post…
Here is how to change the mirrors.

cd /etc/yum.repos.d/
sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-*
sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*
dnf update
cd

There are two packages, dracut-network and rpmconf, that conflict with upgradingand need to be removed.

dnf remove dracut-network rpmconf

Remove the old CentOS 7 kernel

rpm -e `rpm -q kernel`

Remove any conflicting packages that are not needed any longer

rpm -e --nodeps sysvinit-tools

Now run the upgrade for CentOS 8

dnf -y --releasever=8 --allowerasing --setopt=deltarpm=false distro-sync

Next it is time to install a new kernel on your VM. 

dnf -y install kernel-core

The final step to perform is to install CentOS 8 minimal packages

dnf -y groupupdate "Core" "Minimal Install"

Now if you recheck you can see that both the CentOS version and the kernel version have been updated.

Updated CentOS version

Category: CentOS, Linux | LEAVE A COMMENT
1 April 2022

Bitnami Start or Stop Services

I found a great Bitnami Docs KB article describing how to check the status of, and stop/start/restart the services running on your Bitnami instance.

Each Bitnami stack includes a control script that lets you easily check the status of, stop, start and restart services.

These are the commands that you would use. If you use them as-is below it will perform the specified action against all the Bitnami services on your instance.

sudo /opt/bitnami/ctlscript.sh status
sudo /opt/bitnami/ctlscript.sh start
sudo /opt/bitnami/ctlscript.sh stop
sudo /opt/bitnami/ctlscript.sh restart

Or use any of the above against a single service that is running, such as Apache only, by passing the service’s name as an argument after the desired action, such as restart.

sudo /opt/bitnami/ctlscript.sh restart apache

The easiest way to learn the names of the services that are on your Bitnami instance is by simply checking all of their statuses with the status command as it returns the names of all the services on your instance.

sudo /opt/bitnami/ctlscript.sh status
Category: Bitnami, Linux, WordPress | LEAVE A COMMENT
31 March 2022

ESXi Access to resource settings on the host is restricted

I was trying to upload an ova file thru vCenter when I got an error message about a controller type in this particular ova. From prior troubleshooting, I knew that the workaround was to just log directly onto a host and upload the ova directly to the host. I know I had done this before so I was confused for a brief moment when I got the error message:

Access to resource settings on the host is restricted to the server that is managing it: xx.xx.xx.xx.

Okay… so the ESXi host is being managed by vCenter. How do I finagle my upload onto a host? Here’s how…

  • Enable SSH (if it is disabled) on the ESXi host you want to upload/deploy your ova or ovf to.
    • You should know how/where to enable SSH… If you don’t there are plenty of articles you can google.
  • Connect to the host via SSH.
  • We want to stop the service that allows the ESXi host and the vCenter to communicate. To do so we want to run the following commands.

/etc/init.d/vpxa stop
/etc/init.d/hostd restart

  • Deploy your ova or ovf to your host thru the ESXi host’s DCUI WebGUI.
  • After you deploy your virtual machine, restart the VPXA service via SSH on the ESXi host.

/etc/init.d/vpxa start

  • Wait a couple of minutes while the ESXi host and your vCenter re-establish communication between each other.
  • If SSH was previously disabled, re-disable it on your host.
Category: ESXi, vCenter, VMWare | LEAVE A COMMENT
27 March 2022

Monitor Desk Mount

After getting a new desk you need to get some hardware to finish dialing it in how you want. For me, after getting a new desk at home, one of those things that I needed was a way to mount my monitor. I spent more hours reading through product reviews than I’d like to admit, but the end result was something I’m happy with.

So heading into my search I had a few things in mind. First, I’m not against purchasing expensive items if their value is worth it. That said, I did not want to go spending a few hundred dollars on a new mount. I felt like there were so many Chinese knockoffs on the Amazon alone that I could surely find a good deal on something of middle-of-the-road quality. Ideally, I wanted to spend under $100 on it. The second thing I really wanted was a mount that would fit dual monitors The plan was to search for a dual monitor stand so I could have two screens running. The third thing I was looking for was for it to be a clamp-on style mount. Having a child in my house I wanted something that would be secured to the desk. Something that I wouldn’t have to worry about being top-heavy and toppling over if the little one was screwing around and “bumped” the desk too hard.

So now that I laid out my wants, how did I do, and what did I decide to get?

After reading a ton of reviews I decided to shift from a dual monitor to a single monitor stand. Having two individual arms would allow me better flexibility on the screen size. With my current monitors being two different sizes (32″ and 21″) this makes things easy and in the future, I don’t ever have to worry about the monitor size as a limitation.

I had lots of clamp-style mounts to choose from on Amazon. It ended up being pretty easy to find one that I liked. However, one thing that I hadn’t originally considered getting was a reinforcement bracket for the clamp. It helps to distribute the pressure from the mount’s clamp and protects the desk from stress fractures over time. It’s basically just a small steel plate that sits between the desk and the clamp. After looking at my desk and giving it some thought, I figured it’d be worth the few extra bucks it cost.

My final purchase was a fully articulating arm for up to a 32″ monitor. I spent $32.99 on it and I feel like I got a good deal on it considering how high some of the arms can cost. The one I purchased was the MountUp Single Monitor Desk Mount.

I paired that with the WALI Steel Reinforcement Bracket for $16.99

And because nothing can ever go smoothly, the VESA mount holes on the arm and one of my monitors were different sizes. So I ended up getting the Husky Mounts VESA adaptor for it for $10.95. It solved my issue no problem.

So I managed to get a solution that works for me all for $50 per monitor (not including the VESA adaptor). I think that I did alright getting everything I wanted for $100.

What sort of monitor stand or mount do you use?

Category: Gadgets | LEAVE A COMMENT
24 February 2022

Changing Your Password from an RDP Session

So here’s the scenario, you’ve RDP-ed into a server and you want to change your password. You try to hit CRTL+ATL+DEL but instead of it getting sent to the remote computer, it opens on your local machine. Blah! That is not what we want… How do we get to a place where we can change the password for the account that was used in the RDP session?

One way to send it within the RDP session is to launch the on-screen keyboard. To launch it, simply click on the ‘Start Menu’ and type “osk”, then click on the result to open the keyboard. With the OSK on screen, press and hold “CTRL+ALT” on your physical keyboard, and click “DEL” on the virtual keyboard button.

The easiest way to bring up the menu from where you can change your password is to press CRTL+ALT+END in the RDP window. Now if you are RDP-ed from a mac, you’ll need to do a CRTL+ALT+Fn+Backspace or CRTL+ALT+Fn+Right-Arrow to bring up the menu.

Category: Microsoft, Windows 10, Windows 2016, Windows 2019 | LEAVE A COMMENT
2 February 2022

vMotion Greyed Out

Okay… This was a first for me. I was in VCSA and went to vMotion a VM, a totally normal task. However, upon clicking on the VM and trying to run the vMotion, it option was greyed out. WHAT THE HECK!?!
Well, thankfully it ended up being a simple fix. Apparently, this can happen if the VCSA database does not get properly updated after a backup job. Here’s how to re-enable it.

Manually remove entries from the vCenter Mob:

  1. Identify the VM’s MOB ID
    1. Open a web browser and go to the vCenter vSphere Client (HTML5)
    2. In the left hand side inventory, select the affected VM’s object
    3. With the VM selected and highlighted, in the URL bar, review the string and look for “VirtualMachine:vm-xx“

Example: https://<VCSA-FQDN>/ui/#?extensionId=vsphere.core.vm.summary&objectId=urn:vmomi:VirtualMachine:vm-69:7071ab73-c123-4d56-ef78-1234gh56i7jk&navigator=vsphere.core.viTree.hostsAndClustersView

In this example VM’s MOB ID is vm-69.

  1. ​​​​​​Access the Virtual Machine Operations by opening a web browser to: 

https://<VCSA-FQDN>/mob/?moid=AuthorizationManager&method=enableMethods

  1. Login with administrator@yourssodomain.local or an SSO admin user (administrator@vsphere.local)
  2. The below page will appear:
  1. In the first parameter entity value box, replace MOID with the VM’s MOB ID

<!– array start –>
<entity type=”ManagedEntity” xsi:type=”ManagedObjectReference”>vm-69</entity>
<!– array end –>

  1. In the second parameter entity value box, enter this command:

<method>RelocateVM_Task</method>

  1. Click on “Invoke Method” in the lower right corner.
  1. Refresh the vCenter Server web client and the migration option for the VM should now be available. If the issue persists, try restarting all of the vCenter Server services.

Here is VMware’s KB about it.

Category: vCenter, VMWare | LEAVE A COMMENT
28 January 2022

Nutanix LCM – Insufficient space on ESXi scratch disk

I was running into an issue where I could not run the Nutanix LCM Inventory action on a cluster because the scratch disk on an ESXi host was too small. And it seems reasonable to assume that if you’re reading this article, then you too probably have encountered an with the pre-check “test_esxi_scratch_space“.

I’ve seen the issue a few times now, and in my experience, the ESXi host has always just had its scratch disk set to the wrong disk. The first couple of times I saw this, the fix was to update the scratch disk from within the advanced host settings in VCSA. The part I did not like about fixing the issue this way, was that it required a reboot of the host for the setting to take hold. Having to schedule a maintenance period or create downtime is never an ideal solution. Thankfully I learned that there is another way to repoint the scratch disk that requires no downtime and it only requires a few lines of CLI against the ‘problem’ ESXi host.

Start by connecting via SSH to the ESXi host that is having the issue with the scratch disk.

Run the command “ls -ll /scratch” to find which volume is currently set as the scratch disk

root@ESXi# ls -ll /scratch
lrwxrwxrwx    1 root     root            49 May  8 23:40 /scratch -> /vmfs/volumes/5xyzxyz6-dxyzxyzb-1c73-ac1xyzxyz990

Run the “df -h” command to list all of the disks on the host and their sizes

root@ESXi# df -h

Filesystem   Size   Used Available Use% Mounted on
NFS          1.6T   1.4T    127.4G  92% /vmfs/volumes/OS-XXX-Repoxxx
VMFS-5      52.0G   1.1G     50.9G   2% /vmfs/volumes/NTNX-local-ds-17xyzz340111-B
vfat         4.0G  27.6M      4.0G   1% /vmfs/volumes/5xyzxyz-1234xyzz-12xy-1234xyzz1234
vfat       285.8M 205.8M     80.0M  72% /vmfs/volumes/5xyzxyz6-dxyzxyzb-1c73-ac1xyzxyz990
vfat       249.7M 152.6M     97.2M  61% /vmfs/volumes/58xyzxyz-cdxyzxyz-766a-12xyzxyz1226
vfat       249.7M 145.3M    104.4M  58% /vmfs/volumes/b4xyzxyz-80xyzxyz-9bf2-e5xyzxyzf6d0

Now that we have the current scratch disk and a list of the sizes of all the disks, we can check if the scratch volume is indeed set to the volume that is 4GB in size.

In the example above we can see that the volume “/vmfs/volumes/5xyzxyz6-dxyzxyzb-1c73-ac1xyzxyz990” is only 285MB in size. That means that this current volume is far too small. No wonder we’re getting an error.

We want to set our scratch disk to a volume that is 4GB in size. According to the list above that means we want to use the volume “/vmfs/volumes/5xyzxyz-1234xyzz-12xy-1234xyzz1234”. To set the desired scratch disk we’ll use the command “ln -sfn <volume_id> /scratch”.

root@ESXi# ln -sfn /vmfs/volumes/5xyzxyz-1234xyzz-12xy-1234xyzz1234 /scratch

If we recheck what the scratch disk is on our host, we’ll see that it is now set to the proper disk volume.

root@ESXi# ls -ll /scratch

lrwxrwxrwx    1 root     root            49 May  8 23:40 /scratch -> /vmfs/volumes/5xyzxyz-1234xyzz-12xy-1234xyzz1234

Now that the scratch disk is properly configured on the host we can update it in VCSA and be done.

From the Host, go to Configure, then Advanced Systems Settings, and click “Edit”.
Select “ScratchConfig.CurrentScratchLocation” and set it to the same value that you just manually configured on host. Hit “Apply”, and you’ll see that the VCSA now recognizes the newly configured scratch disk.

Well now we’re done, and we didn’t even need to reboot a single physical host! You can read more about this error in Nutanix’s KB article about it.

Category: ESXi, Nutanix | LEAVE A COMMENT
5 January 2022

Reset password on locked-out Domain Admin

Sometimes things happen and a password gets forgotten or lost, or in the worst case it wasn’t updated in your password management tool after it was changed. We’ve likely all had to bug another admin to reset our password for one system or another. It happens. But what happens if you are the lone Domain Admin and lock yourself out? Luckily, there is a way to get back in if you do get locked out.

  • Download the Windows Server 2016 ISO.
  • Attach the ISO to your DC virtual machine.
  • Reboot the VM into the ISO
  • Select: Repair your Computer -> Troubleshoot -> Command Prompt
  • At the command prompt, run the following commands:
cd c:\Windows\System32
ren osk.exe osk.old
copy c:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe osk.exe
  • Reboot the Server.
  • Launch the on-screen keyboard and PowerShell will open
  • At the Powershell prompt, run the following command, replacing <PASSWORD> with the password of your choice:
Net user Administrator <PASSWORD>
  • Revert file changes in your System32 folder, renaming ‘osk.old’ back to ‘osk.exe’.

And there you have it folks, you are now able to log back in with your Domain Admin account. This works because while the DC does not have a local Administrator account, it somehow realizes that and resets the Domain Admin. Yes it is a little bit of black magic fuckery in that regard… But it worked and got you back in, so who are we to complain.

Category: Microsoft, Windows 2012/2012R2, Windows 2016, Windows 2019 | LEAVE A COMMENT