Last time I went through some of the basics about getting the latest patch-set of ODI to connect to a GitHub.com VCS repository. Today I will talk a little about versioning our ODI development in Git.
Navigator
The first point to make is that we can select the objects we wish to version-track in Git, it need not be the whole repository; we could just track an individual mapping, but most likely we would be tracking one or more ODI repository projects or project folders. I don’t know about you, but my preferred development style is to use a single project containing multiple folders to organize my codebase; I also create a ‘development playground’ folder where I can try out ideas. Whether we source track the playground folder is debatable. It is a code asset and should be tracked, however it may not sit well with some aspects of ODI Deployment Archive creation from Git sources; I will write more on deployments next time.
The easiest way of adding ODI content to Git VCS is by right-clicking on the object in the ODI designer navigator and then Version/Add to VCS.
This leads to the Add Objects To Git dialog box. As I mentioned in the previous blog, ODI uses a commit and push model; it is mandatory to provide a commit comment every time you version an object in ODI Git; this dialog box is no exception. As you can see, versioning at the project (or folder) level will allow to include child objects; this can save us work and, more importantly, add multiple items in a single commit.
Include Dependencies will ensure that dependent ODI objects are included in the files pushed to Git, the Regenerate and Version Scenarios option is self-explanatory. In the ODI Studio Navigator Git versioned ODI objects have a small green ‘g’ on the object icon.
Having pushed to Git we can see what is in there! Looking in my local file system’s Git repository clone I see that ODI has generated two folder trees, one for the Master Repository and the other for the Work Repository. The structures of the two repository trees follow the same basic outline as you would see on the designer and topology navigators in ODI Studio.
In my screenshot I have also included a couple of other objects that are not managed by ODI (a file and a folder); most organizations would also keep non-ODI items in the same Git repository, for example DDL scripts, data models, shell scripts, deployment notes, in fact just about any resource that needs versioning — being able to do this is an important plus.
Now that we have ODI source code in Git we can get on with developing and versioning as we go. As previously described, I am not in favor of automatically creating a new version on each save, rather I prefer developer choice regarding the right time to commit and push.
We just right click on the object we want to version in the source navigator to access the Version context menu and then select Create VCS Version…
We always need to provide a commit comment for the save. Note too, that ODI is smart enough not to commit to Git if there have not been any changes.
Note the Version menu is context specific; if we right click on a containeritem such as Folder or Model we get an additional option to Add Non-versioned Objects to VCS…. This does exactly what we expect: it brings up a list of objects in that container that are not currently under version control and allows us to select the ones we need to add to Git.
Other options on any VCS tracked object are Version History, Version Tree, Restore From VCS.
From the version history we can restore a previous version from VCS. As a developer, I find it difficult to remember which commit date is the version that I need to restore. Instead, I prefer to use meaningful commit comments such as Jira ticket number or release number. Of course, with Git we could use these commit comment as part of a workflow triggering other updates to our systems. The version history screen also lets us access a useful 3-way version compare window. This allows us to compare the Git baseline with the selected version and the current state of the ODI repository or to compare 2 selected versions against the baseline. In my example below, I added the filter FILTER1 to the baseline mapping and saved to Git — next I renamed the filter Low_numbers and saved to the ODI repository but not pushed to Git.
Clicking on the ‘Select Version’ button brings up a version history window shown on the right of the screenshot above. Again, I find it easier to use the commit comment rather than the date column to identify versions, especially if we create multiple versions per day.
A version tree is a graphical representation of the Git commits for an object, again with access to the version compare feature. The tree is color-coded with the initial create in green, changes in black and deletes in red. For my example mapping I tested the ability to recover a deleted version and hence I get a red spot representing the delete and a yellow one for the restore version step. Hover the mouse over a dot and a pop-up window shows additional detail on the version. Again, we get direct access to the 3-way compare feature by clicking on the icon in the screen menu bar.
Team Menu
The other place where the developer interacts with the ODI-Git versioning system is on the Team Menu -> VCS Actions. I’ll cover most of these options in part 2 of this blog but for now a few of developer-useful features of VCS Actions.
Pending Changes identifies all of the Git tracked objects that have been saved to the ODI metadata repository but have not yet been pushed to Git.
We can, optionally, push one or more of the changes to Git from the Pending Changes viewer or select a single ODI object and restore it from Git. In fact, I prefer making my new version generation from the pending changes screen — it is a clear interface and allows us to ensure that all of the change is committed in a single operation.
The Add Non-versioned Objects option from the VCS Actions menu is not quite the same as the one reached by right clicking on a folder or project in the Navigator. On the navigator we can add all the child objects and their dependencies; in Team we have the ability to select anything in the repositories. If you are working with fine-grain development of just a project or a couple of folders, this may be too big a selection choice. For most of my initial use cases I would only really want to version the stuff created or customized in the development process.
One final thing I’ll mention now is the ability to bring back deleted objects, yes, sometimes developers need to do that.
In the final part of the new Git features in ODI blog I’ll talk about Tags, Deployments and starting afresh in a repository.
