Linux

Going back to the basics….moving out of Amazon Drive!

As of June 8, 2017, it was announced that when when users try to sign up for Amazon Drive they will not be able to select an unlimited cloud storage option. Instead they can choose either 100 GB for $11.99 per year, or 1 TB for $59.99, with up to 30 TB available for an additional $59.99 per TB. (The prior pricing was unlimited everything for $59.99.) My data came up to about 5TB, which according to their new pricing structure would cost me $300+ (Data is always growing!!).

That is quite costly for just 5TB of storage when I can buy two 8TB drives and have it locally in a RAID configuration or a mirrored set. I shopped around with other popular cloud providers but each and every one of them have some sort of limitations. I decided to purchase two 8TB drives and maintain it locally. 

I found very little help on Google when searching for ways to move out of Amazon Drive with ease. I found a lot of cool little utilities but none were able to do a clean and consistent sync copy/move. It most cases the application would either hang, or incomplete the job.

I tried a lot of tools to get a synced local copy but the process seemed harder and harder. I tried a lot of freeware and shareware utilities as well as those offered by Amazon. I am just listing my personal experiences here so that I can save time for those whole have a similar situation.

Tools I tried:

Amazon Drive Desktop Sync

  •  Horrible transfer speeds +
  • Buggy Software
  • Startup & Resuming files would delay download significantly.

SymLink (MacOS/ Linux)

  • Somewhat works but metadata is lost.

NetDrive

  • Mounts the Amazon Cloud Drive and a Network Drive
  • Constant disconnects + too many app updates
  • Application hangs with large files
  • Service needed to be restarted multiple times to connect with Amazon

Cloudberry Explorer

  • Quirks around Admin Mode.
  • Ghostfiles (0kb) leftover.
  • Acts like an FTP Client but missing a lot features

rClone (Banned)

AllwaySync

  • The Oneway transfer feature is nice but it was taking a long time between files
  • This might have worked if my filebase was a whole lot smaller but failed for larger jobs.

Expandrive

  • Similar to NetDrive but a whole lot stable, but would fail on larger files.

Odrive

  • Horrible interface. Didn’t work most of time.

& a few more applications…. that didn’t work out!

Syncovery

Syncovery was the  winner in my case. This tools was the best in speed and got me an exact copy out from Amazon Cloud drive. It supports resuming! It is available on all platforms. It has a nice layout and can run as a scheduled job!

It took Syncovery literally 2 days to get all of my data downloaded. I was simply amazed at how efficient this tool was working. It maintained a consistent speed. Didn’t lose any metadeta. I ran a file check and all of them checked out 100%.

The trial version worked in my case and I am considering getting the Pro version. It excelled where all other failed. It wasn’t a resource hog and did the job in the first go! Thank you Syncovery!

 

Couple of lessons learned in getting success with all my data downloaded.

  1. Metadata is important especially when dealing with older files. Try not of lose it, as once it is lost there is no going back.
  2. Don’t copy to the same path as the original. Use an external drive and copy it there.
  3. If dealing with a lot of smaller files break them into chunks or batches to avoid application hang
  4. Apart from Syncovery, there were some utilities that might delete the files from Amazon and put them in Trash. Make sure you look there if you notice files missing file. It is most certainly there. I personally didn’t have this issue but some people have reported this with other utilities.
  5. Share your experiences to help out others.

Conclusion

I am in no way promoting a product from Syncovery, but based on my personal experience I found it to be the easiest to move the amount of data I had from Amazon down to my local server.  I am going to sway away from the public cloud space for a while at-least for my personal stuff. Based on the pricing, limitation of file size and types, and amount of data I have, I am still searching for good cloud store. I am evaluating ownCloud for now. If I ever goto a public cloud storage solution again, I am going to try my exit exercise/ strategy prior to bulk upload.

Another strategy people are recommending is hosting all the files in a VM on AWS/ Google/ Azure. My issue there is access cost. If my access is within the VM I am good, but any data I am pulling or accessing out of the VM – I am paying for it!

How to Configure Time Server (NTP Server) on Centos 7?

Prerequisites:

VM/ Physical – It is recommended to have the time server as a physical server, but VMs should be okay depending on your setup and requirement. My virtual lab environment is VMware based so here’s the guide for a VM. Once the CentOS Minimal is installed on the VM make sure to install open-vm-tools.

If you are not conformable with Vi Editor please use install nano for editing.

Installing the time server:

  1. First we need to install NTPd . to do this open command line :

  2. Then open NTP main configuration file for editing:
    If you are planning on hosting your time servers externally like time01.yourdomain.com, time02.yourdomain.com with ports 123 open on the firewall. Each of the time server will be talking to 0.us.pool.ntp.org, 0.us.pool.ntp.org, 0.us.pool.ntp.org, 0.us.pool.ntp.org. Later on you can probably add your time servers to the NTP.org by joining the pool. For my example, I am not using the centos default pool, but i am using the U.S. zone. For internal servers/workstations you can point it to the local ip/dns name for the local time servers.

  3. Then start and enable ntp server:

  4. You need to allow ntp service on firewall(NTP service uses UDP port 123). So run next command:

  5. Verify you ntp service with next command:

Your output should be similar to:

Configure SNMP on an ESXi Host or multiple Hosts

Recently I needed to configure all of our 40 or so ESXi hosts to forward SNMP traps to our corporate monitoring solution. This meant enabling and configuring SNMP on each of the hosts. Naturally, I wrote a script for this as 40 hosts is way too many to do manually.

This article shows you how configure SNMP on an ESXi host manually, via PowerCLI and via host profiles.

Option 1: Manually via Command Line

This is the most boring approach and should really only be used if you only have a few ESXi hosts to do, or if you really like doing things manually 🙂

  1. Start the SSH service on the ESXi host (Configuration >> Software >> Security Profile >> Services)
  2. SSH into host (using putty or something similar)
  3. Run the following to configure SNMP settings, enable SNMP in the firewall and start the SNMP agent:
    Note 1: Replace <COMMUNITY_STRING> with the community string for your monitoring solution.

    Note 2: Replace <TARGET_STRING> with the target string that maps to your environment, in the format of [email protected]/community_string.

    Option 2: Manually via PowerCLI

    Option number 2 is to use PowerCLI to configure SNMP on an ESXi host. The following script is how to do this on a single host. To configure SNMP on an whole bunch of ESXi hosts, see option 3 below.

    Note: Prior to being able to use the script above, ensure you configure the following variable values:

    • <ESXI_HOST> – The FQDN or the IP address of the ESXi host you want to enable SNMP on.
    • <COMMUNITY> – This is the community string you require for your environment (same as in option 1 above).
    • <TARGET> – This is the FQDN or IP address of the target you want to send the SNMP traps to. Note: THIS IS NOT A TARGET STRING as in option 1. In this instance you ONLY need the FQDN or IP address. The @port and the community_string will be added automatically by the Set-VMHostSnmp cmdlet.
    • <PORT> – The port you require SNMP traps to be sent on.

Option 3: Automatically via PowerCLI

If you have to configure SNMP for more than just a handful of ESXi hosts, then it is worth automating the entire process through a PowerCLI script. The logic around enabling SNMP on the ESXi host is the same as in option 2 above, with some additional logic around this to enumerate and complete the process on all ESXi Hosts.

Here is a script that will connect to a vCenter Server, get a list of all ESXi Hosts and then configure SNMP on each ESXi host:

Note: Similar to option 2 above, you will need to configure the following variables first:

  • <COMMUNITY> – This is the community string you require for your environment (same as in option 1 above).
  • <TARGET> – This is the FQDN or IP address of the target you want to send the SNMP traps to.
    Note: THIS IS NOT A TARGET STRING as in option 1. In this instance you ONLY need the FQDN or IP address. The @port and the community_string will be added automatically by the Set-VMHostSnmp cmdlet.
  • <PORT> – The port you require SNMP traps to be sent on.

Option 4: Automatically via Host Profiles

Finally, if you are lucky enough to be running Enterprise Plus licensing, then you will have the ability to use Host Profiles. This allows you to configure SNMP within the host profile and then just apply that profile to all of your ESXi hosts.

Follow these steps to add the SNMP configuration into an existing Host Profile:

  1. From the VI Client, navigate to Management >> Host Profiles
  2. Select the profile you want to add the SNMP settings and click Edit Profile
  3. Expand the SNMP Agent Configuration policy and select SNMP Agent Configuration
  4. In the Configuration Details pane, complete the followng:
    • Enable or Disable agent: Ticked
    • IP/UDP Port: The port you require SNMP traps to be sent on
    • SNMP Community String: The community string for your environment
    • Notification Receiver: The target string that maps to your environment, in the format of [email protected]/community_string
  5. Click OK to save changes
  6. For each ESXi host attach and apply the profile (Note: An ESXi host needs to be in maintenance mode to be able to apply the host profile)

And that concludes how to configure SNMP on a ESXi host.

Source

Linux: CentOS 6 kernel panic+0xa7/0x16f fix

So ran into a view issue for a Linux Host today. Kernel Panic !!!

There could be a number of reasons that you might get this error. The three common reasons may be:

  1. There was a change to selinux config – (that might be an error in the config file)
  2. There are disk errors
  3. The kernel is corrupted

None of the kernels in the GRUB boot menu will boot, you get a kernel panic:

1-kernel-panic

Kernel panic – not syncing: Attempted to kill init!
Pid: 1, comm: init Not tainted 2.6.32-504.3.3.el6.x86_64 #1
panic+0xa7/0x16f
do_exit+0x862/0x870
fput+0x25/0x30
do_group_exit+0x58/0xd0
sys_exit_group+0x17/0x20
system_call_fastpath+0x16/0x1b

Booting in single user mode doesn’t work either.

Here’s how to fix this:

            1. Reboot, and go in the GRUB menu. You have 3 seconds to strike the arrow keys before it will automatically boot the default kernel.2-grub-menu
            2. Select the first line, the default kernel, and press the E key on the keyboard to edit the parameters. You will then see the following.
              3-grub-edit
            3. Use the arrow keys to select the 2nd line, that starts with kernel. Press the E key to change this line, use the arrow keys to go to the end and type a space followed by enforcing=0
              4-grub-edit
            4. Press enter to conform and then press B to boot the system.
              It should boot up fine now.
            5. Choose your solution:

              SELINUX Config FileDisk ErrorsRebuild Kernel

              Disabled for the SELINUXTYPE variable, that’s wrong it has to be for the SELINUX variable. In the screenshot below I show you the correct settings in the config file to disable SELINUX.

              5-selinux-disabled

            6. Reboot.

ESXi 6.0 not detecting BROCADE HBA adapter

Steps:

  1. Make sure HBA is connected on the PCI slot and visible under esx hardware list:
  2. Check if VMKernel can detect any storage via Fibre Channel
    (output will be blank line if HBA driver is missing but HBA appears to be in PCI card determined from step 1)
  3. Search and download the relevant ESXi drivers for HBAthe recommended driver (bfa) version for 82B in ESXi 5.1 is 3.0.0.0
    You can download it from the following URL.
    https://my.vmware.com/web/vmware/details?downloadGroup=DT-ESXi50-BROCADE-bfa-3000&productId=229
  4. Download the driver and install it using following instructions:

New Installation

For new installs, you should perform the following steps:

  1. Copy the VIB to the ESX server.  Technically, you can place the file anywhere that is accessible to the ESX console shell, but for these instructions, we’ll assume the location is in ‘/tmp’. Here’s an example of using the Linux ‘scp’ utility to copy the file from a local system to an ESX server located at 10.10.10.10:
  2. Issue the following command (full path to the VIB must be specified):

In the example above, this would be:

Note: Depending on the certificate used to sign the VIB, you may need to change the host acceptance level.  To do this, use the following command:

Also, depending on the type of VIB being installed, you may have to put ESX into maintenance mode.  This can be done through the VI Client, or by adding the ‘–maintenance-mode’ option to the above esxcli command.

Upgrade Installation

The upgrade process is similar to a new install, except the command that should be issued is the following:

Reboot host.

Now you should have the HBA should the datastores.

“Host SMBus controller not enabled!”

CentOS guest instances in VMware sometimes come up with the boot error message:

This error is being caused because VMware doesn’t actually provide that level interface for CPU access, but Ubuntu try to load the kernel module anyway.

How to fix it:

add the line:

Reboot.

NOTE: for older versions use blacklist i2c_piix4 instead.
NOTE: it works both in VMWare Fusion 5 and 6, and Ubuntu LTS 12.04 and 14.04

How to Install VMware Tools on RHEL 7/CentOS 7

VMware Tools is one of important components for virtual machine (VM) in order get excellent performance. It is a group of utilities that help to enhances the overall performance of the virtual machine’s guest operating system (OS) and improves management of the VM. Without the VMware Tools, guest OS performance will lacks some of the important functionality. Below steps shows how to install the VMware Tools on RHEL 7, CentOS 7 and Oracle Linux 7.

How to Install VMware Tools ?

Option 1 (Recommended by VMware) – VMware support of open-vm-tools

open-vm-tools is an open source implementation of VMware Tools from third parties and contains utilities that enhances virtualization management, administration and functions of the virtual machine in VMware environments. The ultimate goal is to enable the operating system vendors and/or communities and virtual appliance vendors to bundle VMware Tools into their product releases.

Option 2 :

1. Install prerequisites :

2. Attach the vmware tools sofware from vSphere client.

3. Mount the vmware tools package into /mnt :

4. Copy vmware tool packagae into /tmp :
5. Go to /tmp directory and extract the vmware tools package :
6. Go to extracted folder, vmware-tools-distrib :
7. Run vmware-install.pl to start installation :
Example:
8. Once successfully installed, make sure you umount (not unmount- missing the n) back the /mnt :