Quick Background on my current CICS integration project
I’m currently working on a stop-gap legacy integration project that is geared toward data integration between a legacy CICS system and a new 3rd party product. I have 2 major tasks in this project, the first being a web application for bulk data entry into both systems simultaneously and the second being cross-system data transformation and transfer. The integration points between the two systems are expected to support operations for the next 12 months. Past integration projects have been handled through cron’d UNIX scripts, PL/SQL stored procedures, triggers, CICS batch jobs, etc. Past projects also have left a dearth of documentation in their paths.
The current integration approach to the CICS system is pre-defined, so I just have to work with “what is” in that regard. I’ve had the joy of reading COBOL ctl files to complete requirements and design docs. Integration involves FTPing flat files to and fro.
The 3rd party product has a service-based API, but service calls are not chunky at all; so interfacing through that is a bit too wordy for the batch processing that I’ll need to be doing. To make things more fun, I have no ER diagrams for the new system’s DB (which has a couple hundred tables), many entity relationships are not supported by foreign keys, no one seems to know what sequences are used to generate keys, the 3rd party system’s source is closed and lacks technical documentation, I have no customer contact (from XP PoV, I only have a manager and a developer to interface with), my wife left me, and my dog died (okay, maybe not the last two).
In any case, this is proving to be a delightful challenge (ah yes, and I failed to mention that the client wants betas for 5 batch interfaces and a the web application written in 3 weeks). The client wants the solution written in VB.Net and has asked that some “best practices” be introduced into the organization along the way. I’ve been constructing a domain model(s) that can speak between the two systems and believe that I’m making good headway in understanding the business better.
I’ll be blogging (maybe?) on some of my approaches as I work though this project. I’ve done a fair bit of VB6 in the past (distant past), but the VB.Net syntax has felt strange having done so much C# over the last few years (at least now I can relate better to the VB.Net guys’ questions at the local UG meetings). Generics, attributes and reflection look especially weird.
As far as tools, I’ll be using some of the usual suspects, but will probably emphasize my use of NHibernate. I’ve also had my first experience with using LDAP with forms authentication (if anyone wants an example I can do that too). In fact, if there’s anything about this project that may be of interest to you, drop a comment and let me know.
VB.Net oddity of the day – Assignment/Comparison operator
On my current project I’m forced to code in VB.Net. Normally I’m pretty open to other languages, but VB.Net is irritating (more so than VB6 in my opinion). The language syntax is riddled with ambiguity (at least for a c-type guy like myself) and moving from c# to VB can be confusing.
Here’s my latest beef with VB.Net. There’s only one operator for assignment and comparisons. As a result, the compiler handles evaluation of the the language syntax in a subtle, but irritating way.
Here’s some code to illustrate my point, I had something like this in my program:
</FONT>Dim oldValue, currentValue As Integer</P>
oldValue = currentValue = 2
In C# this is cut and dry. The value 2 would be assigned to currentValue and the side effect would be the value assigned, or 2. I’ve long been in the habit of using assignment side effects in C#, but VB looks at this differently. In VB, the value 2 is assigned to currentValue, but the side effect is the comparison of the two values. In this case, currentValue initializes to 0, so the expression evaluates to false (or 0). Therefore, oldValue now equals 0.
Now, if I did this:
</FONT>Dim oldValue, currentValue As Integer</P>
currentValue = 2
oldValue = currentValue = 2
When the statement is evaluated, it will first evaluate the expression currentValue = 2. 2 will be assigned to currentValue and the side effect will be the comparison of currentValue to the assigned value, 2. They are equal, so true will be returned and converted from bool to int as the non-zero value -1.
It’s no biggy, just annoying.
CLOBbered again!
Hey all,
I’m currently working on an NHibernate on Oracle project and ran into the first problem with CLOBs that I’ve had in some time. I was trying to use a property that mapped to a CLOB in a Example instance for a session Criteria object. No dice. Oracle complained saying this:
System.Data.OracleClient.OracleException: ORA-00932: inconsistent datatypes: expected – got CLOB
Well, I guess sometimes you get lucky…and sometimes you just get CLOBbered.
Guess now I have to dig into the NHibernate code if I have time. All other CRUD works nice with Clobs, just Example chokes. Could just be a dialect thing.
Cheers!
How to enlist ADO commands into an NHibernate transaction
Adoption of NHibernate in a legacy environment can be daunting for a number of reasons. Aside from the overhead of becoming proficient with the framework itself, developers are also faced with thousands of lines of working (it’s assumed) code that is already conversing with the system’s data store(s). If complete migration to NHibernate is a prerequisite, then such systems would never make the move. That being said, NHibernate does afford a way to make calls using the plain old ADO.Net API. To be more precise, NHibernate allows users to enlist IDbCommands into NHibernate transactions.
Personally, I’ve only used this feature to work around limitations of NHibernate (yes, although I believe NHibernate is the best thing since sliced bread, I do acknowledge its limitations…unlike <disparaging remark about other OS/Language/Methodology fanatics here>). This feature was necessary for bulk delete operations…example follows:
HQL in NHibernate is handy, but can get you in trouble. Say you fire off an HQL query something like this:
DELETE FROM User WHERE User.Clue = null //not sure offhand of exact HQL syntax
This would delete pretty much all your users, so if you have a few million clueless users in the system NHibernate will load the Users into the first level cache and then delete them. I had a similar situation with a gig I was on not too long ago. Caching the objects before deleting them would not only have taken a long time, it would have brought the server to its knees after the page file was used up. I couldn’t skip the caching stage using NHibernate directly, but I could do something like this:
ISession session = sessionFactory.GetSession();
using(ITransaction transaction = session.BeginTransaction())
{
…NHibernate stuff…
IDbCommand command = new SqlCommand();
command.Connection = session.Connection;
transaction.Enlist(command);
command.CommandText = “delete from User where clue is null”;
command.ExecuteNonQuery();
…more NHibernate stuff…
}
In the using block I could do all sorts of NHibernate stuff yet still do my bulk delete without the caching overhead. ADO commands used in this way are safely tucked into your current NHibernate Transaction context.
This feature can be handy, but keep in mind that NHibernate remains unaware of state changes in the repository. If I had a User collection in memory, I’d have to ignore the previous state after the delete and recall the collection based on my previous find criteria. Stored Procedures could also be used this way, but Ayende blogged not to long ago about an alternative approach (I’m not sure off hand of the version of NHibernate that supports this).
Reflectionesque behavior in JavaScript
Here’s a little something in my continued investigation of JavaScript and its many wonders (man, that sounds strange!)…
Okay, so for today’s topic I’ll look at some ‘reflectionesque’ behavior of JavaScript. As many know, the class Object can be used as an associative array. For instance:
var myvar = new Object();
myvar[“value1”] = 42;
assert(42, myvar[“value1”]); //would be true
What many may not know is that objects will behave as associative array whether we like it or not. Properties (fields or functions) that we add to an object’s prototype will show up in a foreach construct if we iterate over an object. This means that we could iterate over an object to see what’s supported by its prototype/interface…sort of like reflection in .Net. This can be helpful if we want to ensure that a property is supported by a given instance. If it’s not supported we can either inject the property/function at runtime so that the object doesn’t cause any problems or we could bypass behavior that depends on the property. Also, we could take disparate classes that support the same “interface” and apply polymorphic ideas. In the example I show an attempt at getting the value of the ‘name’ property from three different classes.
So that joe has something to play with, I’ve attached a script file called reflectionesque.js (zipped) that illustrates some of this. I’ve also attached a screen shot that shows the output:
How’d he get that image up there?
First off, I’m not much of a blogger so I’m not quite up to speed on the tips and trick of blogging. I saw recently pretty pictures in one of Nellie’s blogs and was instanly green with envy. I wanted pictures too! I tinkered around with the dashboard and blog editor for a while to no avail and finally out of desparation went back his blog entry. I noticed WindowsLiveWriter in the URL and gave a quick search. To say the least, I wish I was aware of the tool in the first place. It facilitates creating blog entries easily with rich content. Just as a simple example, I’ve included and image of one of the more lackluster constellations, Triangulum………now I can include screen shots in discussions….yiiiihaaa!
Term of the Day: Principle Of Least Surprise – Epilogue
Okay, this is funny. Little did I know that while writing “Term of the Day: Principle Of Least Surprise” I was entering the midst of just such a problem.
The entry was written over a 2-3 hour span during compile waits on various tasks. The last hour or so required debugging due to some really strange behavior. I was working at the persistence layer on a data mapper responsible for persisting a complex object, adding support for new composite type that was a few levels down in the object model. Everything looked right, but for some reason my tests kept failing. I finally had to debug and intercept the SQL that was generated to see what was going on.
In the end I found the problem in a class that I wasn’t using directly that was providing the table name. This is okay and common in our persistence framework. What was not okay was that it specified the target database along with the table name, ignoring the connection information that was passed back to the data mapper. The mapper was saving data to the correct database, but was retrieving it from another database. I took out notwhatiwant.dbo from the const defining the table name and ran my tests again…some other tests on code that I wasn’t working on started to fail. It didn’t take too long to clean everything up after that, but it was annoying.
…lol, told you it wasn’t that uncommon
P.S. This is also a great argument for automated unit tests. Resolving this was only quick because of over 5,000 automated unit tests that cover our code. The tests that broke relate to the code base from a separate subsystem that I wasn’t even working on. The need that prompted the addition of the database name to the tableName constant was addressed implicitly and in such a way that limited reuse. Without the tests I probably would have ended up deploying a new bug. We have great tests…that’s why we rock.
Term of the Day: Principle Of Least Surprise
The principle of least surprise (AKA POLS) simply dictates that the interface to any given entity (method, for instance) should exhibit the bevahior that is least suprising to the user/programmer when there are conflicts or ambiguities between members of said interface.
Example:
Let’s say that the following are true…
The following classes exist: Person, Address, People
Person has the following attributes: SSN, Name, Address
Person.SSN is used to identify Person instances
People is a service that manages a set of Person objects
People has to following methods: Add(Person):void, Get(SSN):Person
Now let’s say Joe programmer is using People to describe his blooming social life…
Joe says:
myFriend = new Person()
myFriend.Name = “John”
myFriend.SSN = 9999999999
myFriend.Street = 10101 Victory Lane
myFriend.City = San Mango
myFriend.Address.State = Texas
myFriend.Address.Zip = 93940
Then he says:
friends = new People()
friends.Add(myFriend)
…Now Joe’s friends include a reference to John…
Later Joe wants to write a letter to his friend, so he says:
myFriend = friends.Get(999999999)
If the People class adheres to the POLS, the address of myFriend should be that same as it was when he added it to friends.
Now here’s a scenario that we see on occasion that will give Joe a nasty surprise. The author of People decided that states are implied by zip codes. Instead of bothering with the State value that’s passed into People, it just stores the Zip code and gets the appropriate state based on the Zip repository that People’s author went through so much trouble to create. Unfortunately for Joe, when Joe entered the data he had been reminiscing on his own address in central California and he accidentally entered his own Zip. Now Joe’s lost contact with his only friend (poor Joe).
As a result, myFriend looks like this:
myFriend = new Person()
myFriend.Name = “John”
myFriend.SSN = 9999999999
myFriend.Street = 10101 Victory Lane
myFriend.City = San Mango
myFriend.Address.State = California
myFriend.Address.Zip = 93940
Person’s Add() looks like a setter and the Get() looks like a getter. The interface does nothing to communicate that somewhere in between there is an implied mutation based on Zip. This behavior would therefore violate the POLS.
The example may seem a bit silly, but I’m sure we’ve all run into similar scenarios more than we’d like to admit. This concept isn’t new, just often ignored.
Another great JavaScript article
Sorry all, I’ll probably be over this JavaScript tear soon enough, but now that I’ve given the language another glance it has really caught my eye this time. I can only explain my recent engagement with the language as a total re-discovery. I’ve done some js OO before in the past, but looking at the language completely outside of a web development context has really helped my make a paradigm shift. It’s been kind of like when I was scripting in sed then discovered perl.
Here’s a really good js article: http://simon.incutio.com/slides/2006/etech/javascript/js-reintroduction-notes.html
I couldn’t agree more with the author, Simon Willison, when he says “…JavaScript has a reasonable claim to being the world’s most misunderstood programming language.”
Again, the author focuses on js from a web developer viewpoint, but I really think its potential is undervalued when it comes to using the language as a tool for system tools. I just have to find a better way of managing script libraries…
C# -> JavaScript
Joe had asked me recently if I thought I might want to make a tool that compiles C# into JavaScript. I might want to futs with something like that, but fortunately I don’t have to any time soon. There’s already a tool called script# that does just that. This is a pet project of Nikhil Kothari and is definitely a must see. I plan on messing with this in the near future with the goal of covering client code with nunit testing and rendering the business rules related client side scripts during the build.
Here’s a link to the project: http://projects.nikhilk.net/Projects/ScriptSharp.aspx
subscribe via RSS