...to paraphrase Forest Gump. TDI (aka SDI) is an extraordinarily agile platform for exploration, prototyping and crafting enterprise-strength solutions for data movement and services. However, using it or any other tool does not an agile developer make. Agile development is all about mindset and culture rather than processes and tools. In this age of industrialized and weaponized Agile (the
noun vs. the
adjective, as opined by thought leaders like
Ken Schwaber and
Dave Snowden) that perspective can be lost.
But of course, seeing the world through TDI-colored glasses mean that my view is indeed a bit colored...
TDI is an excellent platform for agile development work, also in teams. I've worked in several projects where multiple developers worked on the same TDI projects. However, sharing the source code (ALs, Components, Scripts, etc.) using something like
git requires a little forethought and setup. The first step is good organization of both source and deployment assets.
As I've noted before, I organize projects in a particular way that has served me well for years:
- A Project Folder (the one that Eclipse gives you in your workspace and that shows up in the CE). In general, apart from running git from this folder, I tend to leave everything being managed by Eclipse alone.
- And a Solution Folder (with the same name as the Project itself and found in the Solution Directory).
- Furthermore, all filepaths in the solution are relative and reference files in the Solution Folder, or its sub-directories. Filepaths also have only forward slashes instead of backslashes.
The advantage of this last item is that solutions run equally well on Linux and Windows, and that you can deploy the solution to a new server by first zipping down the Solution Folder and then unzipping it into the Solution Directory of the target TDI server. Of course, along with any CustomJars files and folders your solution depends on.
As you may have gathered from my description above, there needs to be two git repositories: one for the Project Folder and one for the Solution Folder. The Project Folder is the
source while the Solution Folder holds the
deployment assets, including properties and other support files and folders. Which also means all Property Store filepaths should be relative (and with forward slashes). As noted above, you may have custom jars for your solution. Simply drop them all into a folder , e.g.
CustomJars, in the Solution Directory. You let TDI know about this folder via the
com.ibm.di.userjars property in
solution.properties which is found in your Solution Directory. Be sure and make this a relative path too.
This is the first step. Then comes the tricky bit.
Since TDI source files (*.assemblyline, *.connector. etc.) are all XML encoded, merging changes when two or more people edit the same source asset can be a challenge. For one thing, there are plenty of XML tags in these files that are only for internal use, like
<ModTime>. So you end up having go through the files, comparing known tags and script snippets. Not ideal.
Which is why we kept to a strict git regime: push to the trunk (i.e.
master, with no branches) using pull requests for peer review, and agreement on which files we each were working on. Not ideal, but until we can get a VSCode extension that handles this for us, it's the best you can do. Of course, editors like VSCode make it easier to compare changes and accept/reject as desired.
Once source management (and sharing) is organized, it then comes down to organizing work. Here is where Scrum, the brainchild of Ken Schwaber and Jeff Sutherland, both Agile Manifesto signatories, comes in.
Their original vision of Scrum is as a flexible, adjust-as-you-need methodology that really dictates only a few things:
- Cross-functional, empowered teams
- Short work cycles with clearly defined outputs
- Open and continuous communication
The first point is to limit wait times due to expertise or authority lacking in the team. The second item is so that measurable results are continually produced. And finally, the last point is to make sure that a) any blockers can be quickly swarmed on and dealt with, and b) everyone in the team knows what is happening, current priorities and why.
Now you might be saying, hey wait, traditionally practiced Scrum also defines distinct rolls like product owner, dev team and Scrum Master. These roles may be covered by team members, although for small teams they may necessarily be resources outside the team. If so then it is important to open lines of communication with the people performing these duties - and one that provides for informal comms if possible.
Most of the TDI work I've done over the past couple of decades has been with one to two other individuals. So to augment our team, we enlisted someone with both an understanding of the requirements, and the ability to prioritize these. Typically an architect or project manager, who then becomes our Product Owner. As to the Scrum Master, I've provided agile coaching while we all shared the admin work.
And finally, we build a relationship with the person or people responsible for operating the solution once we deliver it. This gives us DevOps insights into how we make the final solution easier to (re)install (e.g. Disaster Recover), manage (stop & restart) and monitor (logs, audits and other operational output).
Posts
