Get Started

Using Oracle’s Baseline Validation Tool with OBIEE — Part 2

By

This post is part 2 of 4 about using the Baseline Validation Tool (BVT) with Oracle Business Intelligence (OBIEE). Post 1 with an intro to BVT, can be found here.

Obtain the BVT tool

  1. Download the latest version of BVT from Oracle. http://www.oracle.com/technetwork/middleware/bi/downloads/bi-bvt-download-3587672.html
  2. Unzip the download into a directory on your computer. To save yourself typing later on, I recommend not embedding this too far into other folders. I put mine in D:/oracle.bi.bvt
  3. Once unzipped, the tool is ready to use.

Create the XML input file

I started out by creating a folder called Scripts or Configs in my BVT folder. This will allow me to keep all of my configuration scripts organized within the BVT folder.

Why do I have so many configuration files? I create a new file for each catalog folder that I’m testing or each different type of plug-in test I’m using or even for the purpose of why I’m testing such as 12c Upgrade or XYZ Project Validation.

Next, open the textconfig.xml file in a code or text editor and save it in your scripts folder with a new name. I suggest giving it a name corresponding to the folder and the test that you’ll be running with this script. If you don’t have this file, I’ve copied it to the bottom of this blog post.

Deployment Section

Within the config file, we’ll first tackle the Deployment section where we specify the Instances that we’ll be running BVT on. The testconfig file shows two deployments but you can put more than two in here. Say for instance you have a dev, test, and prod environment, you can put all three in so that your file is ready to be run on whichever environment you might desire.

Here’s the definitions and what I changed in this section:

  • Deployment Name — This name is used when you run BVT. Give it a concise yet descriptive name.
  • Analytics URL — This will point to your answers instance. Be sure to include http at the beginning and /analytics at the end
  • XmlpURL — If you’ll be running the BIP test, you’ll need to point this to your BI Publisher instance
  • VaURL — If you’ll be running the VA (Data Visualizer) test, you’ll need to point this to your VA instance
  • UserName — Give a user that will not be restricted in what data is shown, unless you are specifically testing security settings.
  • Password — The password for that user. Yes this is clear text and not encrypted.

A more secure way of providing your password is to use –password when calling the bvt procedure instead of storing your password in your config file. If you type –password without providing the password, you will be prompted to enter it when running the tests.

Tests Section

The Tests section of the XML is where you specify which tests to run and which folder to run them on, along with specific settings for each test. The test name can be found after com.oracle.biee.bvt.plugin in the TestPlugin name tag. By default all of the tests are set to true. To turn each test off, in the TestPlugin tag, set enabled=“false”.

I’ve found it easiest to have a separate script for each test and run each test individually to keep the results separated out and concise.

Each test has a tag for the Parameter name of CatalogRoot. This is where you put the path to the folder you’d like the test to run on. For a folder within the Shared Folders structure, you will put “/shared/<folder name>”. Note that you cannot put multiple folders in this list. If you want two shared folders tested, you either need to run against your whole shared catalog or run two separate tests.

Catalog Plug-In Parameters

The only setting in the config for this test is CatalogRoot where you specify which folder to run this test against.

Reports Plug-In Parameters

  • ReadTimeout: The amount of time to allow each report to run before moving to the next report. This is in milliseconds.
  • CatalogRoot: Which folder to run this test against. This can be set differently for each type of report export (Excel, PDF, or csv) if you’d like. To run on the entire catalog, specify value=”/” (but this is not recommended).
  • ScoreThreshold: For Excel & PDF. This is an algorithm that scores the similarity of the exports. A threshold of 1 is a perfect match.
  • DiffTolerance: For CSV exports. The number of decimal places of difference to tolerate for fluctuations in the data. 0 is no tolerance. 0.1 would flag a difference in one field value greater than 0.1 between the two exports.

UI Plug-In Parameters

  • CatalogRoot: Which folder to run this test against. To run on the entire catalog, specify value=”/” (but this is not recommended).
  • ThreadQueueSize: The number of concurrent browsers to use: 0–50.
  • ThreadTimeout: The amount of time to allow each report to run before moving to the next report. This is in milliseconds.
  • BrowserType: Which browser to use for testing. Leave blank for default browser. Firefox or InternetExplorer.
  • FirefoxLocationInLinus: If using Linux, the location of Firefox to use. Leave blank for default.
  • FindReportSnapshotDifferences: Capture screenshots and comparisons of reports within the specified catalog root folder.
  • FindDashboardSnapshotDifferences: Capture screenshots and comparisons of dashboards within the specified catalog root folder.
  • PromptInputFile: See Dashboard Prompts Input below.
  • ScoreThreshold: This is an algorithm that scores the similarity of the screen captures. A threshold of 1 is a perfect match.

Dashboard Plug-In Parameters

  • ExportDashboardsToXML: If set to true, all of the dashboards in the dashboard prompts file AND all of the dashboards found under the Catalog Root folder will be processed. If false, only the dashboards in the dashboard prompts file will be processed.
  • CatalogRoot: Which folder to run this test against. To run on the entire catalog, specify value=”/” (but this is not recommended).
  • PromptInputFile: The path (if not in the base bvt folder) and name of the file containing the Prompt URLs to each dashboard page. See Dashboard Prompts Input File section below for more details about this file.
  • ExportAllDashboards: Set to true if you want BVT to export and test all dashboards within catalog root as specified in the export. If this is set to false, it will only test the reports in the dashboard prompts input file.
  • DiffTolerance: Set a decimal point that should be tolerated for fluctuations in data. “0” is no tolerance. Suggested values are “0.1”, “0.01”, etc.

BI Publisher Plug-In Parameters

The BI Publisher Plug-in (called BIP in BVT) has two test types: Export all BI Publisher reports in a specific folder or use a parameters input file to specify the BI Publisher reports to test

  • ExportReportsToPDF: If set to true, BIP plug-in will test all of the BI Publisher reports in the Catalog Root specified folder.
  • CatalogRoot: For running the ExportReportsToPDF test. Unlike the rest of the plug-ins, the catalog root for the BIP plug-in is different. This plug-in uses the physical path instead of the virtual path under /shared/. For example, if you are using “/shared/Sample Lite/Published Reporting” for other plug-ins, in the BIP plug-in you would use “/Sample Lite/Published Reporting”.
  • ExportReportsWithParametersToPDF: If set to true, BIP plug-in will use a parameter input file to determine which BIP reports to export and compare.
  • ParametersInputFile: For the ExportReportsWithParametersToPDF test. Similar to the Dashboard Prompts file (described below), this parameter will point to the input file containing links with BIP report parameters populated within the URL.
  • ScoreThreshold: Specified for both tests. This is an algorithm that scores the similarity of the report PDF exports. A threshold of 1 is a perfect match.

VA Plug-In Parameters

These plug-in parameters are nearly identical to the UI plug-in parameters.

  • CatalogRoot: Which folder to run this test against.
  • ThreadQueueSize: The number of concurrent browsers to use: 0–50.
  • ThreadTimeout: The amount of time to allow each report to run before moving to the next report. This is in milliseconds.
  • BrowserType: Which browser to use for testing. Leave blank for default browser. Firefox or InternetExplorer.
  • FirefoxLocationInLinus: If using Linux, the location of Firefox to use. Leave blank for default.
  • FindVAProjectSnapshotDifferences: Capture screenshots and comparisons of VA projects within the specified catalog root folder.
  • ScoreThreshold: This is an algorithm that scores the similarity of the screen captures. A threshold of 1 is a perfect match.

Dashboard Prompts Input File

For the UI plug-in Dashboard Snapshot Differences test or the Dashboard plug-in, you can force BVT to use certain prompt values in your dashboard prompts to ensure consistency when testing. The file should contain URLs to each dashboard to be tested with the parameter values included in the URL. For the Dashboard test, it’s possible to restrict BVT to only run the dashboards that are contained in this file. This is another way that you can better specify the specific dashboard pages to test if you are not able to get specific enough using the Catalog Root parameter.

To create the Dashboard Prompts file:

  1. Open a blank text file in notepad or a text editor.
  2. Open a web browser and navigate to the dashboard page you want to test.
  3. Select the prompt values you’d like to use on the dashboard for the test.
  4. In the upper right corner of the dashboard, click Page Options — Create Prompted Link.
  5. Copy the URL from the browser into the text file on a new line.
  6. Repeat steps 2–5 for each Dashboard Page that you would like to test.
  7. Save the file as DashboardPrompts.txt in your BVT folder. This specific name is not required. You can name this as specific as you want to describe the dashboards that you are specifying within it. When I test, I name this similar to the name of the bvt config file that I am creating for this test.
  8. Specify this dashboardsprompts.txt file (along with any necessary path if not saved in the bvt folder) in the PromptInputFile parameter in your config file.

BI Publisher Parameters Input File

For the BI Publisher plug-in test, you can use a parameters input file to specify the values that should be selected for the parameters in each report that is to be tested. The file should contain URLs to each report to be tested with the parameter values included in the URL.

To create the Parameter Input file:

  1. Open a blank text file in notepad or a text editor.
  2. Open a web browser and navigate to the report you want to test on the xmlpserver. Important! You must use the xmlpserver.
  3. Select the parameter values you’d like on the report for this test.
  4. In the upper right corner of the report, click Action Option — Share Report Link. Choose Current Page.
  5. Copy the URL from the browser into the text file on a new line.
  6. Repeat steps 2–5 for each BIP report that you would like to test.
  7. Save the file as ParametersInput.txt in your BVT folder. This specific name is not required. You can name this as specific as you want to describe the reports that you are specifying within it. When I test, I name this similar to the name of the bvt config file that I am creating for this test.
  8. Specify this ParametersInput.txt file (along with any necessary path if not saved in the bvt folder) in the ParametersInput parameter in your config file.

You do not need to create a parameters URL for each BI Publisher environment that you’ll be running your tests against. As long as the catalog paths match in your testing environments, BVT will figure out the deployment path to each report in each environment.

Results Section

The last section of the config file is the Results section. Here is where you can specify the name of the folder where the results will be stored. The default is “Results”. I recommend changing this to something similar to what you titled the config script where it contains the name of the folder the test will be run on, and the test(s) run. There is also the ability to specify that the folders are timestamped. This is helpful if you will be running the test multiple times and do not want the folder results overwritten. *See my note in Part 3 about why this is nice but also makes the code for comparing results a bit more difficult to create.

<?xml version="1.0" encoding="UTF-8"?>
<TestConfiguration>
<Deployment name="PreUpgrade" description="First Test Target">
  <Server>
  <!-- AnalyticsURL Needs to point to the analytics page -->
  <!-- XmlpURL Needs to point to the BI Publisher page -->
  <!-- ex:<AnalyticsURL>http://servername:9704/analytics</AnalyticsURL> -->
  <!-- ex: <XmlpURL>https://servername/xmlpserver</XmlpURL> -->
  <!-- ex: <VaURL>https://servername/va</VaURL> -->
  <!-- If you are using a test certificate on the server for SSL and want to bypass SSL certificate validation, set the IgnoreSSLCertErrors to true. This setting is strictly for Test environments only -->
  <AnalyticsURL></AnalyticsURL>
  <XmlpURL></XmlpURL>
  <VaURL></VaURL>
  <UserName></UserName>
  <Password></Password>
  <IgnoreSSLCertErrors>false</IgnoreSSLCertErrors>
  </Server>
</Deployment>
<Deployment name="PostUpgrade" description="Second Test Target">
  <Server>
  <!-- AnalyticsURL Needs to point to the analytics page -->
  <!-- XmlpURL Needs to point to the BI Publisher page -->
  <!-- ex: <AnalyticsURL>http://servername:9704/analytics</AnalyticsURL> -->
  <!-- ex: <XmlpURL>https://servername/xmlpserver</XmlpURL> -->
  <!-- ex: <VaURL>https://servername/va</VaURL> -->
  <!-- If you are using a test certificate on the server for SSL and want to bypass SSL certificate validation, set the IgnoreSSLCertErrors to true. This setting is strictly for Test environments only -->
  <AnalyticsURL></AnalyticsURL>
  <XmlpURL></XmlpURL>
  <VaURL></VaURL>
  <UserName></UserName>
  <Password></Password>
  <IgnoreSSLCertErrors>false</IgnoreSSLCertErrors>
  </Server>
</Deployment>
<Tests>
<!-- CAUTION: The CatalogRoot Value for BIP Plugin is different than other plugins-->
<!-- For BIP remove the virtual paths where the other plugins use physical paths-->
<!-- To get the physical path of an item, in the Oracle Anwsers catalog, right click the item and select "Properties"-->
<!-- Ex: For BIP   : "/Sample Lite/Published Reporting"-->
<!-- Ex: For others: "/shared/Sample Lite/Published Reporting"-->
<TestPlugin name="com.oracle.biee.bvt.plugin.catalog" enabled="true">
  <Test name="GetCatalogTest" enabled="true">
    <Parameter name="CatalogRoot" value="/"/>
  </Test>
</TestPlugin>
<TestPlugin name="com.oracle.biee.bvt.plugin.ui" enabled="true">
  <Parameter name="CatalogRoot" value="/"/>
  <!-- how many threads to spawn to load reports, should be greater than 0, less than 50 -->
  <Parameter name="ThreadQueueSize" value="1" />
  <!-- Timeout for loading each report in MSecs should be between 5 secs and 10 mins -->
  <Parameter name="ThreadTimeout" value="60000" />
  <!-- Which BrowserType to use to load reports, Firefox, InternetExplorer. Leave empty for default browser  -->
  <Parameter name="BrowserType" value=""/>
  <!-- Location of Firefox browser to be used, if multiple instances of Firefox browser exists on a Linux operating system machine. Leave empty for default browser location -->
  <Parameter name="FirefoxLocationInLinux" value=""/>
  <Test name="FindReportSnapshotDifferences" enabled="true">
    <!-- ScoreThreshold should be between 0.00 and 1.00  -->
    <Parameter name="ScoreThreshold" value="0.95"/>
  </Test>
  <Test name="FindDashboardSnapshotDifferences" enabled="true">
    <Parameter name="PromptInputFile" value="DashboardPrompts.txt" />
    <Parameter name="ScoreThreshold" value="0.95"/>
  </Test>
</TestPlugin>
<TestPlugin name="com.oracle.biee.bvt.plugin.report" enabled="true">
  <!-- Timeout for loading each report in MSecs should be between 5 secs and 10 mins -->
  <Parameter name="ReadTimeout" value="15000" />
  <Test name="ExportReportsToPDF" enabled="true">
    <Parameter name="CatalogRoot" value="/"/>
    <Parameter name="ScoreThreshold" value="0.95"/>
  </Test>
  <Test name="ExportReportsToExcel2007" enabled="true">
    <Parameter name="CatalogRoot" value="/"/>
    <Parameter name="ScoreThreshold" value="0.95"/>
  </Test>
  <Test name="ExportReportsToCSV" enabled="true">
    <Parameter name="CatalogRoot" value="/"/>
    <!-- Set a limit on the maximum tolerance for fluctuations in  floating point numbers in report data. Ex:"0" for no tolerance. "0.1, "0.001", etc.-->
    <Parameter name="DiffTolerance" value="0"/>
  </Test>
</TestPlugin>
<TestPlugin name="com.oracle.biee.bvt.plugin.dashboard" enabled="true">
  <Test name="ExportDashboardsToXML" enabled="true">
    <Parameter name="CatalogRoot" value="/"/>
    <!-- Paste the prompted links to the following file-->
    <Parameter name="PromptInputFile" value="DashboardPrompts.txt"/>
    <Parameter name="ExportAllDashboards" value="true"/>
    <!-- Set a limit on the maximum tolerance for fluctuations in floating point numbers in report data. Ex:"0" for no tolerance. "0.1, "0.001", etc.-->
    <Parameter name="DiffTolerance" value="0"/>
  </Test>
</TestPlugin>
<TestPlugin name="com.oracle.biee.bvt.plugin.bip" enabled="true">
  <Test name="ExportReportsToPDF" enabled="true">
  <!-- The CatalogRoot Value for BIP Plugin is different than other plugins-->
  <!-- For BIP remove the virtual paths where the other plugins use physical paths-->
  <!-- Ex: For BIP   : "/Sample Lite/Published Reporting"-->
  <!-- Ex: For others: "/shared/Sample Lite/Published Reporting"-->
    <Parameter name="CatalogRoot" value="/"/>
    <Parameter name="ScoreThreshold" value="0.95"/>
  </Test>
  <Test name="ExportReportsWithParametersToPDF" enabled="true">
    <Parameter name="ParametersInputFile" value="ParametersInputs.txt" />
    <Parameter name="ScoreThreshold" value="0.95"/>
  </Test>
</TestPlugin>
<TestPlugin name="com.oracle.biee.bvt.plugin.va" enabled="true">
  <Parameter name="CatalogRoot" value="/"/>
  <!-- how many threads to spawn to load reports, should be greater than 0, less than 50 -->
  <Parameter name="ThreadQueueSize" value="1" />
  <!-- Timeout for loading each report in MSecs should be between 5 secs and 10 mins -->
  <Parameter name="ThreadTimeout" value="60000" />
  <!-- Which BrowserType to use to load reports, Firefox, InternetExplorer. Leave empty for default browser  -->
  <Parameter name="BrowserType" value=""/>
  <!-- Location of Firefox browser to be used, if multiple instances of Firefox browser exists on a Linux operating system machine. Leave empty for default browser location -->
  <Parameter name="FirefoxLocationInLinux" value=""/>
  <Test name="FindVAProjectSnapshotDifferences" enabled="true">
    <Parameter name="ScoreThreshold" value="0.95"/>
  </Test>
</TestPlugin>
</Tests>
<Results>
  <LogPath timestampFolders="false">Results</LogPath>
</Results>
</TestConfiguration>

Related Posts

Scroll to Top