Why TDI (version 2)

Here comes an updated version based on feedback:

---

All organizations want to reduce risk and maximize return on current investments, especially during tough times when spending for new IT dwindles and resources are redirected towards improving existing infrastructure. Given that infrastructures tend to be the result of evolution - in other words, survival of the highest switching cost and not a grand architectural plan - they seldom accommodate adjustment, much less the introduction of new software or services. Furthermore, the risk of failure or miscalculation can equate to disruption in business operations.

Enter the newer agile development methodologies whose mission is to solve problems incrementally, constantly adjusting design to meet uncovered constraints and correcting project goals to meet discovered requirements. A mindset and approach that bridges the need for executive oversight with the realities of technical innovation/renovation. TDI is designed from the bottom-up for this purpose.

That's why TDI is bundled with a growing number of IBM products and being adopted by more and more of business partners: It uniquely "closes the gap" between product functionality and client expectations. Rapid try/test/refine cycles with TDI reduce projects to manageable steps that provide feedback for stakeholders, enabling risk re-assesment and course correction.

---

In particular, thanks to Avery Salmon for stepping up to the plate here :)

TDI Elevator Pitch

I have been asked many times to write a blurb for a TDI event (e.g. a LabJam) or conference session, with specific instructions to not use much techie-speak. After numerous iterations, and much careful listening - particularly to our architect, Johan Varno - I'm relatively pleased with the following Why TDI? pitch:

---

Most organizations want to reduce risk, especially during tough times when investment in new IT dwindles and resources are redirected towards improving existing infrastructure. Given that infrastructures tend to be the result of evolution -- in other words, survival of the highest switching cost -- and that most evolutionary steps are painful, there is resistance to change.

Enter the agile development methodologies whose mission is to deliver value in stages so ROI can begin sooner and stakeholder feedback shapes each step. A mindset and approach that bridges the need for executive oversight with the uncertain realities of technical innovation/renovation. TDI is built for this purpose.

That's why TDI is bundled with a growing number of IBM products and being adopted by more and companies and integration professionals. It's a customization and deployment tool that uniquely 'closes the gap' between a product's feature list and client expectations. TDI's rapid try/test/refine approach helps reduce complex projects to manageable sprints, enabling continual risk re-assesment and course correction.

---

Comments, suggestions and even ridicule is anxiously awaited :)

Dealing with Errors

Back from summer holiday and starting to remember what I do for a living again. I thought I'd start with a quick post about error handling.

As you already know, the How-To writeup here talks about the mechanics of TDI error flows and Hooks, and from the Configs I've received, people are at the very least enabling Error Hooks so that their AssemblyLines don't stop. A good start.

However, logging enough information to manually recover from a problem is even better.

So what is enough information? That all depends on what your AL is doing and where the error occurs. If the AssemblyLine fails then you get a Java stack dump preceeded with the component name in brackets, along with the operation and underlying error returned from the target system. However if you catch the error by enabling Error Hooks then you don't see this in the log. This information is still available though and you can find it in the error variable.

Just like work and conn, error is an Entry object and it contains a number of Attributes provided by the TDI Server. These are described in the online docs and you can access them - for example, to write to log output, or to implement handling logic - using the getString() method. You can also do a task.dumpEntry(error) to easily get all the information printed to the log.

Sometimes an exception is not an error, but instead an unexpected situation like finding no match for a Lookup or Delete, or finding multiple entries (Lookup, Delete or Update modes). This results in mandatory Hooks like On No Match or On Multiple Found being invoked, and just like with Error Hooks, you have to tell TDI how to proceed or the AL stops. In these cases it is your Link Criteria that is either too narrow or too broad. As a result you will want to log enough information to determine why this is happening, and dumping out error will not get you there.

Instead, you should print out the value of work Attributes used for your Link Criteria: the right-hand operand for simple Link Criteria, usually preceded with a dollar sign ($). That way you can identify which data entries are giving you problems.

For those cases where a lookup is expected to either find no match or multiple then another approach is to add a Loop (Connector Loop) with your Connector attached. The Loop will then cycle once for each Entry found, driving these to components under this Loop. No Hook coding required.

Finally, in the case of connection errors you should note that all Connectors have a Connection Error tab where you can enable the Auto-Reconnect feature. While this is generally not interesting behavior for an initialization failure, if comms drops in the middle of AL processing, Auto-Reconnect can be used to quietly re-establish the connection and your AssemblyLine continues processing as if nothing had happened.

Note that for Iterators this will result in the Select being re-issued as well. As long as Iterator state is maintained - done automatically for all Change Detection Connectors, and you can easily implement this yourself for most other Connectors - processing will resume where it left off. If state is not used to re-align the Iterator then your AL will be reading from the start again.