/
Automate check-in of development OTBI catalog items to SCM

Automate check-in of development OTBI catalog items to SCM

Oracle OTBI projects can source catalog objects either from a source code repository or from development OTBI instance. Use "Discover from Git" (or another source control management system) or "Discover from OTBI Web Catalog" on your FlexDeploy Package-based project used for building and deploying WebCatalog objects. Once catalog objects are discovered (or rediscovered), you can create Package with specific catalog objects for build and deploy.

The OTBI plugin modifies the file extensions and paths of exported or downloaded objects to establish metadata for each object. As a result, simply downloading the .catalog files from the OTBI catalog manager and placing them in source control is not a viable option, as it would be incompatible with the build and deployment operations of the OTBI plugin. To streamline this process, FlexDeploy offers OTBI export and download operations, eliminating the need for manual manipulation of files and ensuring they follow the correct structure and extensions.

There are two options, see below for more details. Using the exportWebCatalog and downloadWebCatalog plugin operations allows you to synchronize OTBI WebCatalog objects from a development instance to a source code repository. By configuring your build and deploy project to source from the repository rather than an OTBI instance, you gain control over when objects become available and eligible for build and deployment. This ensures more flexibility and management over the deployment process.

exportWebCatalog

This process is managed through a package within a Package-based deployment project. Users can create a package as needed, or it can be managed via an incoming webhook. It's important to note that this is a separate project from the one used for build and deployment, and its sole purpose is to synchronize files from the development server to Git or any other source control management (SCM) system.

Ensure project classification is Package-based and project type is Oracle Transactional BI. Project SCM type should be None. Your project for this purpose may look something like this. 

As with other projects, you will configure project properties to specify the folder in the catalog that you will be working with. The goal is to extract files from the development server, and then commit the modified files to Git. Note that you are using this as part of build workflow.

Now let's talk about Extract WebCatalog workflow. This workflow will execute 6 steps as shown below.

  1. Clone the git repository where objects are stored.

  2. Extract WebCatalog files into local git repository folder. Extract will be generally done from development environment. Number of objects extracted will depend on whether All Files or Package build is executing.

  3. Add any new files in the local git repository.

  4. Commit any changes to local git repository. It is possible that nothing is found to commit if there were no changes on development server.

  5. Push changes to remote repository.

Here is source code for the build workflow. You will need to update the steps in the workflow according to your Git configuration (e.g. Git Instance, sparse checkout folder, etc.). Also, keep in mind that we are running this build against the master branch (stream), if you want to commit on a different branch then create the stream on the project as appropriate.

We are referencing Git in the example above, but plugins exist for many other popular source control management systems as well and a similar workflow can be created to accomplish the same task.

name: Extract WebCatalog description: '' steps: - id: '1' name: Git Clone type: INVOKE_PLUGIN data: pluginName: FlexagonGITPlugin pluginOperation: clone inputs: - name: FDGIT_INP_INSTANCE_CODE value: value: FLEXTEST isExpression: false isEncrypted: false - name: FDGIT_INP_DEST_SUBFOLDER value: value: ../obieefiles isExpression: false isEncrypted: false - name: FDGIT_INP_BRANCH value: value: main isExpression: false isEncrypted: false - name: FDGIT_INP_DEPTH value: value: '1' isExpression: false isEncrypted: false - name: FDGIT_INP_SPARSE_CHECKOUT_FOLDERS value: value: '' isExpression: false isEncrypted: false endpointInstanceOverride: isExpression: false consumesArtifacts: false producesArtifacts: false endpointSelection: choice: All endpointExecution: choice: Any outputs: [] userInputs: [] userOutputs: [] - id: '2' name: exportWebCatalog type: INVOKE_PLUGIN data: pluginName: FlexagonOTBIPlugin pluginOperation: exportWebCatalog inputs: - name: FDOTBI_INP_EXPORT_FOLDER value: value: ../obieefiles/OBIEECatalog isExpression: false isEncrypted: false - name: FDOTBI_INP_COPY_PERMISSIONS value: value: 'true' isExpression: false isEncrypted: false - name: FDOTBI_INP_PRESERVE_TIMESTAMPS value: value: 'false' isExpression: false isEncrypted: false endpointInstanceOverride: isExpression: false consumesArtifacts: false producesArtifacts: false endpointSelection: choice: All endpointExecution: choice: Any outputs: [] userInputs: [] userOutputs: [] - id: '3' name: Add any new Files type: INVOKE_PLUGIN data: pluginName: FlexagonGITPlugin pluginOperation: add inputs: - name: FDGIT_INP_INSTANCE_CODE value: value: FLEXTEST isExpression: false isEncrypted: false - name: FDGIT_INP_DEST_SUBFOLDER value: value: ../obieefiles isExpression: false isEncrypted: false - name: FDGIT_INP_FILELIST value: value: '--all .' isExpression: false isEncrypted: false endpointInstanceOverride: isExpression: false consumesArtifacts: false producesArtifacts: false endpointSelection: choice: All endpointExecution: choice: Any outputs: [] userInputs: [] userOutputs: [] - id: '4' name: Commit to Git type: INVOKE_PLUGIN data: pluginName: FlexagonGITPlugin pluginOperation: commit inputs: - name: FDGIT_INP_INSTANCE_CODE value: value: FLEXTEST isExpression: false isEncrypted: false - name: FDGIT_INP_DEST_SUBFOLDER value: value: ../obieefiles isExpression: false isEncrypted: false - name: FDGIT_INP_NAME value: value: FD_WF_USER isExpression: true isEncrypted: false - name: FDGIT_INP_EMAIL value: value: FD_WF_USER_EMAIL isExpression: true isEncrypted: false - name: FDGIT_INP_REVISION_MESSAGE value: value: OBIEE files added from flexdeploy workflow. isExpression: false isEncrypted: false endpointInstanceOverride: isExpression: false consumesArtifacts: false producesArtifacts: false endpointSelection: choice: All endpointExecution: choice: Any outputs: [] userInputs: [] userOutputs: [] - id: '5' name: Push to Central Repository type: INVOKE_PLUGIN data: pluginName: FlexagonGITPlugin pluginOperation: push inputs: - name: FDGIT_INP_INSTANCE_CODE value: value: FLEXTEST isExpression: false isEncrypted: false - name: FDGIT_INP_DEST_SUBFOLDER value: value: ../obieefiles isExpression: false isEncrypted: false - name: FDGIT_INP_BRANCH value: value: main isExpression: false isEncrypted: false - name: FDGIT_INP_FOLLOW_TAGS value: value: 'false' isExpression: false isEncrypted: false endpointInstanceOverride: isExpression: false consumesArtifacts: false producesArtifacts: false endpointSelection: choice: All endpointExecution: choice: Any outputs: [] userInputs: [] userOutputs: []

downloadWebCatalog

This operation also exports or downloads objects from the OTBI WebCatalog. However, unlike the package-based deployment project, this operation does not require such a project and can instead be called directly from within a utility workflow.

Ensure project classification is Utility. Your project for this purpose may look something like this:

Now let's talk about Download WebCatalog workflow. This workflow will execute 6 steps as shown below.

  1. Clone the git repository where objects are stored.

  2. Download WebCatalog files into the local git repository folder. The download will be generally from a development environment. All files from the specified path will be downloaded.

  3. Add any new files in local git repository.

  4. Commit any changes to the local git repository. It is possible that nothing is found to commit if there were no changes on development server.

  5. Push changes to the remote repository.

Here is source code for the utility workflow. You will need to update this workflow to indicate project Git instance code and sparse checkout folder for catalog objects:

We are referencing Git in the example above, but plugins exist for many other popular source control management systems as well, and a similar workflow can be created to accomplish the same task.

For Git, if you want the files placed in a sub-directory rather than the root of the repository, be sure to set the following on your workflow:

  • Clone step - Sparse Checkout Folder

  • downloadWebCatalog step - Download Folder (value should be same as Sparse Checkout Folder)

name: downloadWebCatalogs description: '' inputs: - code: INPUT_WEB_CATALOG_ITEMS displayName: Web Catalog Items dataType: String subDataType: TextField defaultValueExpression: false listDataExpression: false multiselect: false required: false encrypted: false hideWhen: type: equals negate: false values: [] disableWhen: type: equals negate: false values: [] steps: - id: '1' name: Git Clone type: INVOKE_PLUGIN data: pluginName: FlexagonGITPlugin pluginOperation: clone inputs: - name: FDGIT_INP_INSTANCE_CODE value: value: FLEXTEST isExpression: false isEncrypted: 'false' - name: FDGIT_INP_DEST_SUBFOLDER value: value: ../otbifiles isExpression: false isEncrypted: 'false' - name: FDGIT_INP_BRANCH value: value: main isExpression: false isEncrypted: 'false' - name: FDGIT_INP_TREELESS_CLONE value: value: 'false' isExpression: false isEncrypted: 'false' - name: FDGIT_INP_DEPTH value: value: '1' isExpression: false isEncrypted: 'false' - name: FDGIT_INP_SPARSE_CHECKOUT_FOLDERS value: value: '' isExpression: false isEncrypted: 'false' endpointInstanceOverride: isExpression: false consumesArtifacts: false producesArtifacts: false endpointSelection: choice: All endpointExecution: choice: Any outputs: [] userInputs: [] userOutputs: [] - id: '2' name: downloadWebCatalog type: INVOKE_PLUGIN data: pluginName: FlexagonOTBIPlugin pluginOperation: downloadWebCatalog inputs: - name: FDOTBI_INP_OBJECT_PATH value: value: INPUT_WEB_CATALOG_ITEMS isExpression: true isEncrypted: 'false' - name: FDOTBI_INP_DOWNLOAD_FOLDER value: value: ../otbifiles/OTBICatalog isExpression: false isEncrypted: 'false' - name: FDOTBI_INP_COPY_PERMISSIONS value: value: 'true' isExpression: false isEncrypted: 'false' - name: FDOTBI_INP_PRESERVE_TIMESTAMPS value: value: 'false' isExpression: false isEncrypted: 'false' - name: FDOTBI_INP_DISCOVER_SUB_ITEMS value: value: Scorecard, Dashboard Group isExpression: false isEncrypted: 'false' endpointInstanceOverride: isExpression: false consumesArtifacts: false producesArtifacts: false endpointSelection: choice: All endpointExecution: choice: Any outputs: [] userInputs: [] userOutputs: [] - id: '3' name: Add any new Files type: INVOKE_PLUGIN data: pluginName: FlexagonGITPlugin pluginOperation: add inputs: - name: FDGIT_INP_INSTANCE_CODE value: value: FLEXTEST isExpression: false isEncrypted: 'false' - name: FDGIT_INP_DEST_SUBFOLDER value: value: ../otbifiles isExpression: false isEncrypted: 'false' - name: FDGIT_INP_FILELIST value: value: '--all .' isExpression: false isEncrypted: 'false' endpointInstanceOverride: isExpression: false consumesArtifacts: false producesArtifacts: false endpointSelection: choice: All endpointExecution: choice: Any outputs: [] userInputs: [] userOutputs: [] - id: '4' name: Commit to Git type: INVOKE_PLUGIN data: pluginName: FlexagonGITPlugin pluginOperation: commit inputs: - name: FDGIT_INP_INSTANCE_CODE value: value: FLEXTEST isExpression: false isEncrypted: 'false' - name: FDGIT_INP_DEST_SUBFOLDER value: value: ../otbifiles isExpression: false isEncrypted: 'false' - name: FDGIT_INP_NAME value: value: FD_WF_USER isExpression: true isEncrypted: 'false' - name: FDGIT_INP_EMAIL value: value: FD_WF_USER_EMAIL isExpression: true isEncrypted: 'false' - name: FDGIT_INP_REVISION_MESSAGE value: value: OTBI files added from flexdeploy workflow. isExpression: false isEncrypted: 'false' endpointInstanceOverride: isExpression: false consumesArtifacts: false producesArtifacts: false endpointSelection: choice: All endpointExecution: choice: Any outputs: [] userInputs: [] userOutputs: [] - id: '5' name: Push to Central Repository type: INVOKE_PLUGIN data: pluginName: FlexagonGITPlugin pluginOperation: push inputs: - name: FDGIT_INP_INSTANCE_CODE value: value: FLEXTEST isExpression: false isEncrypted: 'false' - name: FDGIT_INP_DEST_SUBFOLDER value: value: ../otbifiles isExpression: false isEncrypted: 'false' - name: FDGIT_INP_BRANCH value: value: main isExpression: false isEncrypted: 'false' - name: FDGIT_INP_FOLLOW_TAGS value: value: 'false' isExpression: false isEncrypted: 'false' endpointInstanceOverride: isExpression: false consumesArtifacts: false producesArtifacts: false endpointSelection: choice: All endpointExecution: choice: Any outputs: [] userInputs: [] userOutputs: []

 

The following macros are not currently supported in the footer:
  • style