A few months back one of my colleagues and good friend Dominic did a presentation about NSX PowerOPS. Due to a busy schedule, I was not able to try it and test it up until last week.
Business Problem
NSX has transformed customer’s approach to Networking & Security and has achieved a significant installed base. Operations is challenged with Day-2 understanding, documentation & health of their NSX deployments.
The main business problem that arises here is that one has all the tools suggested to operationalize NSX but the team is still at a loss when they have to use them to troubleshoot NSX”
For who is this
The NSX Power Operations (Tool) is for people who are responsible for the installation or administrators of the NSX environment.
PowerOPS - What is it and what does it do
The NSX Power Operations (Tool) is a tool that is Day-2 focused tool to extend product usefulness & adoption, Automated documentation, focusing on daily operational activities.
The installation
The installation steps are outlined below.
Check version of PowerShell
Make sure version 5 or above of PowerShell is installed, and if it is installed already you can verify this with the following command:
PS C:\Users\Administrator> $PSVersionTable
Name Value---- -----PSVersion 5.1.14393.0PSEdition DesktopPSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}BuildVersion 10.0.14393.0CLRVersion 4.0.30319.42000WSManStackVersion 3.0PSRemotingProtocolVersion 2.3SerializationVersion 1.1.0.1
PS C:\Users\Administrator>Here you see that I already had version 5.x installed of PowerShell.
Change Execution Policy
PS C:\Users\Administrator> Set-ExecutionPolicy RemoteSigned
Execution Policy ChangeThe execution policy helps protect you from scripts that you do not trust. Changing the execution policy might exposeyou to the security risks described in the about_Execution_Policies help topic athttp://go.microsoft.com/fwlink/?LinkID=135170. Do you want to change the execution policy?[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "N"): APS C:\Users\Administrator>Install PowerCLI
PS C:\Users\Administrator> Find-Module VMware.PowerCLI A quick verification tells us that PowerCLI in installed correctly.
PS C:\Users\Administrator> Find-Module VMware.PowerCLI
Version Name Repository Description---- ---- ---------- -----------10.2.0.... VMware.PowerCLI PSGallery This Windows PowerShell module contains VMware.Po...We can also verify what VMware PowerCLI modules are now available.
PS C:\Users\Administrator> Get-Module VMware* -ListAvailable
Directory: C:\Users\Administrator\Documents\WindowsPowerShell\Modules
ModuleType Version Name ExportedCommands---- ------- ---- ----------------Script 6.7.0.8... VMware.DeployAutomation {Add-DeployRule, Add-ProxyServer, Add-ScriptBundle, Copy-De...Script 6.7.0.8... VMware.ImageBuilder {Add-EsxSoftwareDepot, Add-EsxSoftwarePackage, Compare-EsxI...Manifest 10.2.0.... VMware.PowerCLIScript 6.7.0.9... VMware.VimScript 10.1.0.... VMware.VimAutomation.Cis.Core {Connect-CisServer, Disconnect-CisServer, Get-CisService}Script 10.0.0.... VMware.VimAutomation.Cloud {Add-CIDatastore, Connect-CIServer, Disconnect-CIServer, Ge...Script 10.1.0.... VMware.VimAutomation.CommonScript 10.1.0.... VMware.VimAutomation.Core {Add-PassthroughDevice, Add-VirtualSwitchPhysicalNetworkAda...Script 6.5.4.7... VMware.VimAutomation.HA Get-DrmInfoScript 7.5.0.8... VMware.VimAutomation.HorizonView {Connect-HVServer, Disconnect-HVServer}Script 10.0.0.... VMware.VimAutomation.License Get-LicenseDataManagerScript 10.2.0.... VMware.VimAutomation.Nsxt {Connect-NsxtServer, Disconnect-NsxtServer, Get-NsxtService}Script 10.0.0.... VMware.VimAutomation.PCloud {Connect-PIServer, Disconnect-PIServer, Get-PIComputeInstan...Script 10.1.0.... VMware.VimAutomation.SdkScript 10.0.0.... VMware.VimAutomation.Srm {Connect-SrmServer, Disconnect-SrmServer}Script 10.1.0.... VMware.VimAutomation.Storage {Add-KeyManagementServer, Copy-VDisk, Export-SpbmStoragePol...Script 1.2.0.0 VMware.VimAutomation.StorageUtility Update-VmfsDatastoreScript 10.1.0.... VMware.VimAutomation.Vds {Add-VDSwitchPhysicalNetworkAdapter, Add-VDSwitchVMHost, Ex...Script 10.0.0.... VMware.VimAutomation.Vmc {Connect-Vmc, Disconnect-Vmc, Get-VmcService, Connect-VmcSe...Script 10.0.0.... VMware.VimAutomation.vROps {Connect-OMServer, Disconnect-OMServer, Get-OMAlert, Get-OM...Script 6.5.1.7... VMware.VumAutomation {Add-EntityBaseline, Copy-Patch, Get-Baseline, Get-Complian...Download GIT and Download PowerOPS with using a command windows not powershell
Download GIT from this website Download PowerOPS with using a regular command window (NOT PowerShell).
C:\Users\Administrator>git clone https://github.com/vmware/nsx-powerops.gitCloning into 'nsx-powerops'...remote: Counting objects: 849, done.remote: Compressing objects: 100% (36/36), done.remote: Total 849 (delta 23), reused 28 (delta 13), pack-reused 800RReceiving objects: 100% (849/849), 1.14 MiBVerify if the git repository is cloned properly.
C:\Users\Administrator>dir Volume in drive C has no label. Volume Serial Number is BA5F-76FD
Directory of C:\Users\Administrator
09/18/2018 04:19 AM <DIR> .09/18/2018 04:19 AM <DIR> ..09/18/2018 02:19 AM <DIR> Contacts09/18/2018 02:19 AM <DIR> Desktop09/18/2018 02:34 AM <DIR> Documents09/18/2018 04:12 AM <DIR> Downloads09/18/2018 02:19 AM <DIR> Favorites09/18/2018 02:19 AM <DIR> Links09/18/2018 02:19 AM <DIR> Music09/18/2018 04:19 AM <DIR> nsx-powerops09/18/2018 02:19 AM <DIR> Pictures09/18/2018 02:19 AM <DIR> Saved Games09/18/2018 02:19 AM <DIR> Searches09/18/2018 02:19 AM <DIR> Videos 0 File(s) 0 bytes 14 Dir(s) 89,933,549,568 bytes free
C:\Users\Administrator>Run the PowerOPS script after the installation
PS C:\Users\Administrator\nsx-powerops> .\nsx-PowerOps.ps1When you run PowerOPS for the first time you can set up “Connection Profiles” with hostnames and log in credentials for different componenenets. Once these are initially set these can be changed later on in the menu if required.
Initial startup menu
NSX-PowerOps Version: 2.5.*************************************************************************************************************************
NSX-PowerOps Main Menu
1 - Install NSX-PowerOps Dependancies [ Installed ]2 - Connection Profiles [ Default: 10.11.11.122 ]3 - Configure Scheduled Tasks [ ENABLED ]4 - NSX-PowerOps Documentation Tools [ ENABLED ]5 - NSX-PowerOps HealthChecks [ ENABLED ]
*************************************************************************************************************************Default Connection Profile: 10.11.11.122Current Connection Profile: 10.11.11.122Connected: TrueOutput Directory: C:\Users\Administrator\nsx-powerops\Report\2018-09-18_23-56*************************************************************************************************************************
Enter the number of your choice, h to get item specific help, x to go back or q to quit.Select 4 for NSX - PowerOps Documentation Tools
NSX-PowerOps Version: 2.5.*************************************************************************************************************************
NSX-PowerOps Main Menu > NSX-PowerOps Documentation Tools
Auto Run All1 - Run All Following Documentation Menu Items [ ENABLED ]
Environment Documentation2 - Document All NSX Components [ ENABLED ]3 - Document ESXi Host(s) Info [ ENABLED ]
Networking Documentation4 - Document NSX Environment via Visio Diagramming Tool [ ENABLED ]5 - Document Routing information [ ENABLED ]6 - Document Load Balancing Information [ ENABLED ]
Security Documentation7 - Document NSX DFW info to Excel via DFW2Excel (small environments) [ ENABLED ]8 - Document NSX DFW info to Excel via DFW2Excel (large environments) [ ENABLED ]
*************************************************************************************************************************Default Connection Profile: 10.11.11.122Current Connection Profile: 10.11.11.122Connected: TrueOutput Directory: C:\Users\Administrator\nsx-powerops\Report\2018-09-18_23-56*************************************************************************************************************************
Enter the number of your choice, h to get item specific help, x to go back or q to quit.Select 2 - Document All NSX Components
Generated File by PowerOPS 2._NSX-Components-2018-09-19_00-06.xlsx
Select 3 - Document ESXi Host Info
Output errors after selecting the option
out-event : A specified parameter was not correct: argument[0]At C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1:426 char:17+ out-event -entrytype error $ErrorMessage+ 16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST) + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,out-event
Exception calling "Add" with "2" argument(s): "Item has already been added. Key in dictionary: 'VXLAN Installed' Key being added: 'VXLAN Installed'"At C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1:427 char:17+ ... $sshCommandOutputDataLogicalSwitch.Add("VXLAN Installed", ...+ 16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)~~ + CategoryInfo : NotSpecified: (:) [], MethodInvocationException + FullyQualifiedErrorId : ArgumentException
out-event : A specified parameter was not correct: argument[0]At C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1:426 char:17+ out-event -entrytype error $ErrorMessage+ 16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST) + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,out-event
Exception calling "Add" with "2" argument(s): "Item has already been added. Key in dictionary: 'VXLAN Installed' Key being added: 'VXLAN Installed'"At C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1:427 char:17+ ... $sshCommandOutputDataLogicalSwitch.Add("VXLAN Installed", ...+ 16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)~~ + CategoryInfo : NotSpecified: (:) [], MethodInvocationException + FullyQualifiedErrorId : ArgumentException
out-event : A specified parameter was not correct: argument[0]At C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1:426 char:17+ out-event -entrytype error $ErrorMessage+ 16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST) + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,out-event
Exception calling "Add" with "2" argument(s): "Item has already been added. Key in dictionary: 'VXLAN Installed' Key being added: 'VXLAN Installed'"At C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1:427 char:17+ ... $sshCommandOutputDataLogicalSwitch.Add("VXLAN Installed", ...+ 16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)~~ + CategoryInfo : NotSpecified: (:) [], MethodInvocationException + FullyQualifiedErrorId : ArgumentExceptionThe errors displayed above are probably due to a compatibility problem of the combination of the version PowerNSX and the version NSX that I am using. I did not try to troubleshoot these as the goal of this exercise only was to demonstrate how the PowerOPS script worked.
Generated File by PowerOPS 3._ESXi-Hosts-2018-09-19_00-07.xlsx
Select 4 - Document NSX Environment via Visio Diagramming Tool
Output after selecting the option
PowerNSX Object Capture Script - Version 2.0.1
Getting NSX Objects Getting LogicalSwitches Getting DV PortGroups Getting VSS PortGroups Getting Logical Routers Getting Edges Getting NSX Controllers Getting VMs Getting IP and MAC details from Spoofguard Getting configured IP Set objects Getting configured Services Getting configured Service groups Getting configured Security groups Getting configured Security tags Getting configured DFW Rules Getting Security Policies
Creating Object Export Bundle
Capture Bundle created at C:\Users\Administrator\nsx-powerops\Report\2018-09-18_23-56\NSX-CaptureBundle-2018-09-19_00-10.zipPowerNSX Object Diagram Script
Launching Microsoft Visio.
Building Diagram Adding DLR-01 to diagram with stencil Logical Router Adding LS-TRANSIT to diagram with stencil Logical Switch Connecting Logical Router with Logical Switch with text: uplink: 192.168.10.2 Adding LS-PRODUCTION-01 to diagram with stencil Logical Switch Connecting Logical Router with Logical Switch.4 with text: internal: 172.16.10.1 Adding LS-DHCPTEST to diagram with stencil Logical Switch Connecting Logical Router with Logical Switch.6 with text: internal: 172.16.11.1 Adding LS-MARCEL to diagram with stencil Logical Switch Connecting Logical Router with Logical Switch.8 with text: internal: 10.11.30.1 Adding RoutedNetwork-92c01f5a-e699-4568-8726-e1161fdf0963 to diagram with stencil Logical Switch Connecting Logical Router with Logical Switch.10 with text: internal: 172.16.30.129 Adding RoutedNetwork-5213d324-af25-4fa1-a7c1-a19771fc08af to diagram with stencil Logical Switch Connecting Logical Router with Logical Switch.12 with text: internal: 172.16.30.161 Adding LS-PRODUCTION-02 to diagram with stencil Logical Switch Connecting Logical Router with Logical Switch.14 with text: internal: 172.16.12.1 Adding ESG-01 to diagram with stencil Edge Adding VLAN16-vDS-NSX to diagram with stencil PortGroup Connecting Edge with PortGroup with text: uplink: 10.11.16.1 Connecting Edge with Logical Switch with text: internal: 192.168.10.1 Adding Edge-DemoApp-99f68d14-4bb1-4a81-9594-62f73e6529fb to diagram with stencil Edge Connecting Edge.20 with Logical Switch.12 with text: uplink: 172.16.30.162 172.16.30.163
Saved diagram at C:\Users\Administrator\nsx-powerops\Report\2018-09-18_23-56\NSX-CaptureBundle-2018-09-19_00-10.vsdxDone. Hit any key to continue.Generated Files by PowerOPS 4._NSX-CaptureBundle-2018-09-19_00-10.vsdx
Select 5 - Document Routing information
Generated File by PowerOPS 5._NSX-Routing-2018-09-19_00-12.xlsx
Select 6 - Document Load Balancing Information
Output errors after selecting the option
You typed an invalid name for a sheet or chart. Make sure that:• The name that you type does not exceed 31 characters.• The name does not contain any of the following characters: : \ / ? * [ or ]• You did not leave the name blank.At C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1:972 char:9+ $wsStats.Name = $edge.name+ 16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)~ + CategoryInfo : OperationStopped: (:) [], COMException + FullyQualifiedErrorId : System.Runtime.InteropServices.COMExceptionThe errors displayed above are probably due to a compatibility problem of the combination of the version PowerNSX and the version NSX that I am using. I did not try to troubleshoot these as the goal of this exercise only was to demonstrate how the PowerOPS script worked.
Generated File by PowerOPS 6._LBToExcel-2018-09-19_00-13.xlsx
Select 7 - Document NSX DFW info to Excel via DFW2Excel small environments
Output after selecting the option
Collection of IP Addresses EnabledCollection of Security Tag VM Membership EnabledCollection of Security Group VM Membership Enabled
Retrieving IP Addresses for ALL Virtual Machines in vCenter environment.*** This may take a while ***.
Retrieving Services configured in NSX-v.
Retrieving Service Groups configured in NSX-v.
Retrieving MACSETS configured in NSX-v.
Retrieving IPSETS configured in NSX-v.
Retrieving Security Groups configured in NSX-v.
Retrieving Security Groups Effective Membership in NSX-v.Collection of VM Sec Membership Enabled
Retrieving Security Tags configured in NSX-v.
Retrieving VMs in DFW Exclusion List
Retrieving DFW Redirect FW Rules
Retrieving DFW Layer 2 FW Rules
Retrieving DFW Layer 3 FW Rules
Retrieving Environment SummaryGenerated File by PowerOPS 7. DfwToExcel-2018-09-19_00-15.xlsx
Select 8 - Document NSX DFW info to Excel via DFW2Excel large environments
This output would be the same as option 7.
Select x to go back and select 5 for the health checks
Select 5 for NSX - PowerOps HealthChecks
NSX-PowerOps Version: 2.5.*************************************************************************************************************************
NSX-PowerOps Main Menu > NSX-PowerOps HealthChecks
Non-Interactive Healthchecks Menu1 - NSX Connectivity Test [ ENABLED ]2 - NSX Manager Test [ ENABLED ]3 - NSX Controllers Appliance Test [ ENABLED ]4 - NSX Logical Switch Test [ ENABLED ]5 - NSX DFW Memory Test (requires SSH to ESXi) [ ENABLED ]6 - Check VIB Version [ ENABLED ]
Interactive Healthchecks Menu7 - Check DLR Instance [ ENABLED ]8 - Check vTEP to vTEP connectivity [ ENABLED ]
Beta Healthchecks Menu9 - Compare DLR and Hosts Routing Tables [BETA] [ ENABLED ]
*************************************************************************************************************************Default Connection Profile: 10.11.11.122Current Connection Profile: 10.11.11.122Connected: TrueOutput Directory: C:\Users\Administrator\nsx-powerops\Report\2018-09-18_23-56*************************************************************************************************************************
Enter the number of your choice, h to get item specific help, x to go back or q to quit.Select 1 - NSX Connectivity Test
Output after selecting the option
Using ESXi credentials from connection profile 10.11.11.122Using NSX Controller credentials from connection profile 10.11.11.122Using NSX Manager credentials from connection profile 10.11.11.122
Describing Basic System Connectivity and Time TestsNSX Manager: 10.11.11.122 [+] is connected to vCenter 585ms [-] is within 5 minutes of the local system time (Note: Local system and NSX Manager must be in same TimeZone) 165ms Expected {419.986275675} to be less than {5} 49: $totalAbsMinsDrift The errors displayed above are probably due to a compatibility problem of the combination of the version PowerNSX and the version NSX that I am using. I did not try to troubleshoot these as the goal of this exercise only was to demonstrate how the PowerOPS script worked.
Generated File by PowerOPS TestNSXConnections-2018-09-19_00-24.xml
Select 2 - NSX Manager Test
Output after selecting the option
Using ESXi credentials from connection profile 10.11.11.122Using NSX Controller credentials from connection profile 10.11.11.122Using NSX Manager credentials from connection profile 10.11.11.122
Describing NSX Manager [+] performed an Inventory less than 30 minutes ago 648ms [+] has the NSX Manager Service in a running state 5.59s [+] has the vPostgres Service in a running state 81ms [+] has the RabbitMQ Service in a running state 22ms [-] has the NSX Replicator Service in a stopped state 150ms Expected: {} to match the expression {STOPPED} 75: $_.name -match 'NSX Replicator'}).status The errors displayed above are probably due to a compatibility problem of the combination of the version PowerNSX and the version NSX that I am using. I did not try to troubleshoot these as the goal of this exercise only was to demonstrate how the PowerOPS script worked.
Generated File by PowerOPS TestNSXManager-2018-09-19_00-26.xml
Select 3 - NSX Controllers Appliance Test
Output after selecting the option
Using ESXi credentials from connection profile 10.11.11.122Using NSX Controller credentials from connection profile 10.11.11.122Using NSX Manager credentials from connection profile 10.11.11.122
Describing NSX ControllersThe NSX controller cluster [-] has the supported number of controller nodes 4.14s Expected: {3} But was: {1} 54: It "has the supported number of controller nodes" {$NsxControllers.Count The errors displayed above are probably due to a compatibility problem of the combination of the version PowerNSX and the version NSX that I am using. I did not try to troubleshoot these as the goal of this exercise only was to demonstrate how the PowerOPS script worked.
Generated File by PowerOPS TestNSXControllers-2018-09-19_00-33.xml
Select 4 - NSX Logical Switch Test
Output after selecting the option
Using ESXi credentials from connection profile 10.11.11.122Using NSX Controller credentials from connection profile 10.11.11.122Using NSX Manager credentials from connection profile 10.11.11.122
Describing Logical Switchesthe Logical Switch LS-TRANSITLogical Switch control plane tests [-] is found on all controllers 2.56s String lengths are both 26. Strings differ at index 13. Expected: {VNI found on 1 controllers} But was: {VNI found on 0 controllers} ------------------------^ 115: "VNI found on $($CurrLsVniResults.count) controllers" The errors displayed above are probably due to a compatibility problem of the combination of the version PowerNSX and the version NSX that I am using. I did not try to troubleshoot these as the goal of this exercise only was to demonstrate how the PowerOPS script worked.
Generated File by PowerOPS TestNSXLogicalSwitch-2018-09-19_00-28.xml
Select 5 - NSX DFW Memory Test 〈requires SSH to ESXi〉
Output after selecting the option
Using ESXi credentials from connection profile 10.11.11.122Using NSX Controller credentials from connection profile 10.11.11.122Using NSX Manager credentials from connection profile 10.11.11.122
Describing Distributed Firewall Memory heaps [+] esx-02.home.local is reachable via ssh 1.55s [+] esx-02.home.local returns system heaps 158ms [+] esx-02.home.local has not exceeded the 80% memory threshold on memory heap vsip-attr-0x4322ab5a3000/ for esx-02.home.local 86ms [+] esx-02.home.local has not exceeded the 80% memory threshold on memory heap vsip-flow-0x4310336f6000/ for esx-02.home.local 115ms [+] esx-02.home.local has not exceeded the 80% memory threshold on memory heap vsip-ipdiscovery-0x4310136f3000/ for esx-02.home.local 76ms [+] esx-02.home.local has not exceeded the 80% memory threshold on memory heap vsip-rules-0x4323f77a0000/ for esx-02.home.local 70ms [+] esx-02.home.local has not exceeded the 80% memory threshold on memory heap vsip-state-0x430f736f0000/ for esx-02.home.local 86ms [+] esx-02.home.local has not exceeded the 80% memory threshold on memory heap vsip-module-0x430ef36ed000/ for esx-02.home.local 71ms [+] esx-01.home.local is reachable via ssh 199ms [+] esx-01.home.local returns system heaps 139ms [+] esx-01.home.local has not exceeded the 80% memory threshold on memory heap vsip-attr-0x43251f631000/ for esx-01.home.local 73ms [+] esx-01.home.local has not exceeded the 80% memory threshold on memory heap vsip-flow-0x4310ec78d000/ for esx-01.home.local 85ms [+] esx-01.home.local has not exceeded the 80% memory threshold on memory heap vsip-ipdiscovery-0x430f1fae6000/ for esx-01.home.local 73ms [+] esx-01.home.local has not exceeded the 80% memory threshold on memory heap vsip-rules-0x43245f82e000/ for esx-01.home.local 84ms [+] esx-01.home.local has not exceeded the 80% memory threshold on memory heap vsip-state-0x43104c78a000/ for esx-01.home.local 69ms [+] esx-01.home.local has not exceeded the 80% memory threshold on memory heap vsip-module-0x430fcc787000/ for esx-01.home.local 78ms [+] esx-04.home.local is reachable via ssh 226ms [+] esx-04.home.local returns system heaps 137ms [+] esx-04.home.local has not exceeded the 80% memory threshold on memory heap vsip-attr-0x430f0df72000/ for esx-04.home.local 88ms [+] esx-04.home.local has not exceeded the 80% memory threshold on memory heap vsip-flow-0x430c1a068000/ for esx-04.home.local 77ms [+] esx-04.home.local has not exceeded the 80% memory threshold on memory heap vsip-ipdiscovery-0x430979eeb000/ for esx-04.home.local 78ms [+] esx-04.home.local has not exceeded the 80% memory threshold on memory heap vsip-rules-0x430eae06f000/ for esx-04.home.local 69ms [+] esx-04.home.local has not exceeded the 80% memory threshold on memory heap vsip-state-0x430e6e06c000/ for esx-04.home.local 87ms [+] esx-04.home.local has not exceeded the 80% memory threshold on memory heap vsip-module-0x430dee069000/ for esx-04.home.local 70msTests completed in 3.85sPassed: 24 Failed: 0 Skipped: 0 Pending: 0 Inconclusive: 0Generated File by PowerOPS TestNSXDistributedFirewallHeap-2018-09-19_00-36.xml
Select 6 - Check VIB Version
Output after selecting the option
Using ESXi credentials from connection profile 10.11.11.122Using NSX Controller credentials from connection profile 10.11.11.122Using NSX Manager credentials from connection profile 10.11.11.122
[-] Error occurred in test script 'C:\Users\Administrator\nsx-powerops/HealthCheck/testNSXVIBVersion.Tests.ps1' 640ms NotSupportedException: The response content cannot be parsed because the Internet Explorer engine is not available, or Internet Explorer's first-launch configuration is not complete. Specify the UseBasicParsing parameter and try again. at <ScriptBlock>, C:\Users\Administrator\nsx-powerops\HealthCheck\testNSXVIBVersion.Tests.ps1: line 72 at <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\Pester\3.4.0\Pester.psm1: line 297 at Invoke-Pester, C:\Program Files\WindowsPowerShell\Modules\Pester\3.4.0\Pester.psm1: line 310 at runNSXTest, C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1: line 1322 at <ScriptBlock>, C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1: line 1715 at Show-MenuV2, C:\Users\Administrator\nsx-powerops\util.ps1: line 328 at <ScriptBlock>, C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1: line 1667 at Show-MenuV2, C:\Users\Administrator\nsx-powerops\util.ps1: line 328 at <ScriptBlock>, C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1: line 1915 at <ScriptBlock>, <No file>: line 1Tests completed in 640msPassed: 0 Failed: 1 Skipped: 0 Pending: 0 Inconclusive: 0The errors displayed above are probably due to a compatibility problem of the combination of the version PowerNSX and the version NSX that I am using. I did not try to troubleshoot these as the goal of this exercise only was to demonstrate how the PowerOPS script worked.
Generated File by PowerOPS TestNSXVIBVersion-2018-09-19_00-37.xml
Select 7 - Check DLR Instance
Output after selecting the option
Using ESXi credentials from connection profile 10.11.11.122Using NSX Controller credentials from connection profile 10.11.11.122Using NSX Manager credentials from connection profile 10.11.11.122
Please provide the Compute Cluster name [example: Compute Cluster A1]: ComputePlease provide Cluster's VDS name [example: ComputeA_VDS]: vDS-ComputePlease provide VDN ID [example: 10000]: 5001
Describing NSX VDR Port TestsCluster ComputeWARNING: The Get-EsxCli cmdlet provides a new interface to the ESXCLI functionality. Use the -V2 parameter to switch to the new cmdlet interface. Check the cmdlet help for more information.
Scripts that use the old cmdlet interface may not be compatible across two different versions of the ESX server. The old cmdlet interface is deprecated and will be removed in a future version.Checking Host: esx-02.home.localChecking Host: esx-01.home.localChecking Host: esx-04.home.local [+] got VDR deployed 322.1sTests completed in 322.1sPassed: 1 Failed: 0 Skipped: 0 Pending: 0 Inconclusive: 0Generated File by PowerOPS TestNSXVDR-2018-09-19_00-38.xml
Select 8 - Check vTEP to vTEP connectivity
Output after selecting the option
Using ESXi credentials from connection profile 10.11.11.122Using NSX Controller credentials from connection profile 10.11.11.122Using NSX Manager credentials from connection profile 10.11.11.122
Describing NSX Manager
>> Please provide the MTU size to test [Default: 1572]:
>> Run this test from 'one' host or 'all' [Default: all]:
****************************** Pinging From Host: esx-01.home.local Pinging From VMKnic: vmk3 Pinging From VMKnic IP: 10.11.15.7 Ping MTU Size is: 1572 ******************************
Pinging To Host: esx-01.home.local Pinging To its VMKnic: vmk3 Pinging To its IP: 10.11.15.7 [+] Ping test result 17.25s [+] round-trip min/avg/max=0.041/0.044/0.049ms
Pinging To Host: esx-02.home.local Pinging To its VMKnic: vmk3 Pinging To its IP: 10.11.15.6 [+] Ping test result 2.9s [+] round-trip min/avg/max=0.25/0.274/0.299ms
Pinging To Host: esx-04.home.local Pinging To its VMKnic: vmk3 Pinging To its IP: 10.11.15.8 [+] Ping test result 2.7s [+] round-trip min/avg/max=0.367/0.426/0.49ms
****************************** Pinging From Host: esx-02.home.local Pinging From VMKnic: vmk3 Pinging From VMKnic IP: 10.11.15.6 Ping MTU Size is: 1572 ******************************
Pinging To Host: esx-01.home.local Pinging To its VMKnic: vmk3 Pinging To its IP: 10.11.15.7 [+] Ping test result 2.79s [+] round-trip min/avg/max=0.189/0.234/0.268ms
Pinging To Host: esx-02.home.local Pinging To its VMKnic: vmk3 Pinging To its IP: 10.11.15.6 [+] Ping test result 2.65s [+] round-trip min/avg/max=0.045/0.062/0.072ms
Pinging To Host: esx-04.home.local Pinging To its VMKnic: vmk3 Pinging To its IP: 10.11.15.8 [+] Ping test result 2.73s [+] round-trip min/avg/max=0.414/0.436/0.451ms
****************************** Pinging From Host: esx-04.home.local Pinging From VMKnic: vmk3 Pinging From VMKnic IP: 10.11.15.8 Ping MTU Size is: 1572 ******************************
Pinging To Host: esx-01.home.local Pinging To its VMKnic: vmk3 Pinging To its IP: 10.11.15.7 [+] Ping test result 2.81s [+] round-trip min/avg/max=0.26/0.304/0.37ms
Pinging To Host: esx-02.home.local Pinging To its VMKnic: vmk3 Pinging To its IP: 10.11.15.6 [+] Ping test result 3.74s [+] round-trip min/avg/max=0.414/0.448/0.494ms
Pinging To Host: esx-04.home.local Pinging To its VMKnic: vmk3 Pinging To its IP: 10.11.15.8 [+] Ping test result 2.93s [+] round-trip min/avg/max=0.109/0.115/0.12msTests completed in 40.49sPassed: 9 Failed: 0 Skipped: 0 Pending: 0 Inconclusive: 0Generated Files by PowerOPS VMKnicPingTestOutput-2018-09-19_00-46.xlsx TestNSXMTUUnderlay-2018-09-19_00-45.xml
Select 9 - Compare DLR and Hosts Routing Tables BETA
Output after selecting the option
Using ESXi credentials from connection profile 10.11.11.122Using NSX Controller credentials from connection profile 10.11.11.122Using NSX Manager credentials from connection profile 10.11.11.122
Using existing PowerCLI connection to 10.11.11.52Found single logical router DLR-01
Collecting routing table from DLR-01Collecting routing table from host esx-02.home.localCollecting routing table from host esx-01.home.localCollecting routing table from host esx-04.home.local
Validating routing table of ESXi server esx-02.home.local [-] Error occurred in test script 'C:\Users\Administrator\nsx-powerops/HealthCheck/testNSXDlrHostRoutingTable.Tests.ps1' 4.38s RuntimeException: The It command may only be used inside a Describe block. at Assert-DescribeInProgress, C:\Program Files\WindowsPowerShell\Modules\Pester\3.4.0\Functions\Describe.ps1: line 125 at ItImpl, C:\Program Files\WindowsPowerShell\Modules\Pester\3.4.0\Functions\It.ps1: line 140 at It, C:\Program Files\WindowsPowerShell\Modules\Pester\3.4.0\Functions\It.ps1: line 117 at <ScriptBlock>, C:\Users\Administrator\nsx-powerops\HealthCheck\testNSXDlrHostRoutingTable.Tests.ps1: line 287 at <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\Pester\3.4.0\Pester.psm1: line 297 at Invoke-Pester, C:\Program Files\WindowsPowerShell\Modules\Pester\3.4.0\Pester.psm1: line 310 at runNSXTest, C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1: line 1322 at <ScriptBlock>, C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1: line 1736 at Show-MenuV2, C:\Users\Administrator\nsx-powerops\util.ps1: line 328 at <ScriptBlock>, C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1: line 1667 at Show-MenuV2, C:\Users\Administrator\nsx-powerops\util.ps1: line 328 at <ScriptBlock>, C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1: line 1915 at <ScriptBlock>, <No file>: line 1Tests completed in 4.38sPassed: 0 Failed: 1 Skipped: 0 Pending: 0 Inconclusive: 0The errors displayed above are probably due to a compatibility problem of the combination of the version PowerNSX and the version NSX that I am using. I did not try to troubleshoot these as the goal of this exercise only was to demonstrate how the PowerOPS script worked.
Generated File by PowerOPS TestNSXDlrHostRoutingTable-2018-09-19_00-47.xml
Originally published on the old wiki at /index.php/The_NSX_PowerOPS_Tool_unleashed.