test-featuredYesterday VMware released the vRealize Production Test which is an add-on tool to the vRealize Suite that you can use to monitor the production health of your vRealize Suite deployment from the initial deployment to the operational phase. By using vRealize Production Test, you can run a suite of tests from the command line using the API of the vRealize Suite components to assess and report on the health of those components.

The vRealize Suite is a platform of data collection intensive applications used to govern and manage virtual and cloud infrastructure.  In a production environment outages can range from loss of connectivity to one vCenter, to a complete inability to provision or monitor the environment. vRealize Production Test is a solution for monitoring the vRealize suite ecosystem including the external components that the suite interacts with.

By deploying vRealize Production Test operators of the vRealize Suite are able to continuously test their environment for adherence to best practices and known issues that have already been remedied via knowledgebase articles. vRealize Production Test provides notifications of problems arising before or at the same time that end users are impacted allowing a timely resolution.

The vRealize Production Test software is distributed as a standard Java jar file that can be used as the foundation for monitoring dependencies between the vRealize Suite and the broader ecosystem.  In a medium complexity SDDC design there are many components outside of the suite itself that are crucial to the proper operation of the suite.  Because each of these deployments are tuned to the specific business needs, vRealize Production Test is the ideal foundation for managing the overall “here and now” health of the SDDC.  Building on the core vRealize context aware framework, it tests for the integration between vCenter, NSX, and the customer’s CMDB/IPAM implementation.

vRealize Production Test: Product support

For now it supports the following products and versions:

  • vRealize Automation 6.2.x.
  • vRealize Operations 6.0.x.

vRealize Production Test: Results

  • Concrete remediation steps for issues identified by out of the box content.
  • Extensibility to support deployments from small to custom SDDC implementations.
  • Integration with common infrastructure monitoring components, continuous execution/testing.

vRealize Production Test: Configuration

vRealize Production Test is a Java 7 JAR file that you download to and run on a machine that is outside of the vRealize Suite environment.

The vRealize Production Tool provides a simple user interface for generating a configuration definition. Once the configuration definition is created, it is used as input to the execution of test tool. Below is an example of the configuration needed to set up the tool for vRealize Automation, vRealize Operations and vRealize Business Standard.

Note: The passwords entered are encrypted before they are saved in the configuration file.

The configuration tool is started using the following command line:

java -jar vrealize-readiness-1.0.0.jar -config

Once loaded, the configuration is quit simple, there are three separate configuration tabs for vRealize Automation, vRealize Operations, and vRealize Business. That’s all!

vRPT-vRA config

vRPT-vROPS config

vRPT-vRB config

After entering your environment attributes, you need to save the configuration file by the Save button on each product configuration page. You will need to configure an output directory, under Settings, User Preferences, before you will be allowed to save your configurations. This is a required parameter when you execute the test in the next step.

vRPT-Output config

The vRealize Production Tool is executed from the command line and directs feedback both to the console and to log files. Most of the time, you will likely want to execute all of the available tests by choosing the “-oobList” parameter. It is possible to specific exactly which tests you want to execute, but that will be covered more in another article.

The following command will execute all of the out-of-box tests:

java -jar vrealize-readiness-1.0.0.jar -run -oobList vRA -configDir 

All test results will be printed to the screen as well as recorded in log files located in the “test-output” folder.

Once you have executed the vRealize Production Tool, basic test reports are generated in the test-output sub-folder, based on where the execution occurred. The following reports are generated:

  • test-output/index.html – HTML report page.
  • test-output/testng-results.xml – Machine readable full output.
  • test-output/testng-failed.xml – Machine readable failure report.
  • test-output/juintreports/* – Failure reports in JUnit format.
  • allure-results – Allure XML reports

The results from these test can be evaluated in in a number of ways, and then used to generate automated alerts related to the health of your vRealize tools. As an example, this tool is in use for the VMware Hands On Labs which use vRealize Automation, to ensure all service started up correctly when the vPOD boots up. During the startup, we execute the tests from Power CLI and the output is examined for failures, like this:

vRPT - Reporting

vRealize Production Test: What is tested?

vRealize Automaton

  • Certificates – ensure all certificates for all components are valid for all nodes.
  • Endpoints – test connectivity to endpoints and initial data collection.
  • Networks – tests connectivity between all vRA nodes.
  • Reservations – sanity tests for all reservations and verifying compute resources.
  • Service Status – confirms all CAFE services are properly registered.
  • VAMI tests – verifies the configuration of the virtual appliance(s), including time synchronization, licenses and passwords.

vRealize Operations

  • Sanity tests – basic validation of the vRealize Operations environment.
  • Adapter Instances – test collection status for vROps instances.
  • Solutions – check for pre-installed solutions.
  • Cluster Deployments – verify status of related nodes of a cluster.
  • Inventory Tree Filters – verify vCenter adapter tree filters are loaded.
  • vCenter Inventory Objects – verify inventory sync with vCenter.
  • Resource Metrics – verify a variety of metrics being collected.
  • Certificate chain – test around SSL certificates between nodes in a cluster.

vRealize Business Standard

  • Sanity tests – basic validation of the vRealize Business environment.
  • System Health Status – validate sync between vRB and vCenter.