Geeks With Blogs

News
Charles Young

In all these months of living in BizTalk 2004 land, it is only recently that I have had occasion to do something serious with the BizTalk mapper - and I am not impressed.   I had a requirement to transform an input consisting of multiple records which use attributes to capture field values into an output where, for each input record, the transform creates two similar, but different, records, and then an arbitrary sequence of additional records for those fields in the input which have a value.   Bit of a mouthful, that, but the point is that the output structure was quite different to the input structure.

I merrily created a map to do the work.  I worked out I needed to employ the Table Looping and Table Extractor functoids.   After a bit of playing around, I began to get the desired results for a subset of the record attributes.   The last thing I needed to do to complete my map was to repeat a part of my functoid design that implemented some conditional logic for about 20 or so remaining attributes in the input.

Disaster struck.  On adding a 32nd functoid to my map (is this number significant?), Visual Studio began to take several minutes to compile the map.   As I added more functoids, the time went up dramatically until, very soon, the compiler would run for half an hour or so and then simply report an out-of-memory exception.

A quick visit to the Microsoft knowledgebase confirms that this is currently a known problem with complex maps.  There is a hotfix apparently, and the issues should go away in a future service pack release.   However, in this case, the problem turned out to be a blessing in disguise.   When I looked at the XSLT created by the mapper (you can find this under the temporary folder in your profile) I got a shock. For my complex map the XSLT is truly awful!  You just don't want stuff like this is your nice, carefully crafted BizTalk solution.   I understand why the XSLT is so bad.   The mapper is precisely what it says it is - a mapper between records, fields and functoids.  It doesn't directly represent the XSLT template-based pattern-matching world to the developer, and is constrained in how well it can exploit XSLT.  It is simply not suited to complex scenarios, especially where the output has a significantly different structure to the input.

I re-wrote the map using raw XSLT (you can use a hand-crafted XSLT to define a BizTalk map).  I know XSLT well, and it took me significantly less time to do this that it did to work out how to use the mapper to get the same results.   The XSLT is compact and efficient (the map works significantly faster than the original incomplete map created via the mapper), and is frankly easier to maintain, even though I have no graphical representation any longer.   The moral of this story - even if Microsoft fixes the problems with the mapper, don't use it for complex transforms.

Posted on Monday, August 2, 2004 1:52 AM BizTalk Server 2004/2006 | Back to top


Comments on this post: BizTalk Server 2004: Using the mapper for complex transforms - don't!!

# re: BizTalk Server 2004: Using the mapper for complex transforms - don't!!
Requesting Gravatar...
Hi,

Good points there, I think I need to try this...

>>you can use a hand-crafted XSLT to define a BizTalk map

Any pointers on how to do this??

Cheers,

Alan

Left by Alan on Aug 02, 2004 3:38 PM

# re: BizTalk Server 2004: Using the mapper for complex transforms - don't!!
Requesting Gravatar...
After mucking around with the mapper for a little while now I've come to a slightly different conclusion (although who knows when I'll change my mind again).

In most of the transformations that I've worked on, there is a combination of some straight-forward mapping (which the mapper makes very easy) and some more complex mapping (which the mapper makes very hard or impossible). So, I choose (on an output record by output record basis) whether to do things with the mapper or just to insert a Scripting functoid (of type "Inline XSLT") and process it as I would in a hand-crafted XSLT.

This has a couple of advantages:
1) On those records that map easily, you can very quickly do the mapping. This saves on typing (and the possibility of typos).
2) Those simple mappings are probably easier to read than looking at hand-crafted XSLT. (With hand crafted XSLT, you need to look at it closely enough to see that there's nothing fancy going on; that is immediately obvious in the mapper).

For the complex bits, the Scripting functoids are pretty much the XSLT that you would have written anyhow.

The down side is that you need to know more to use this approach. You need to know everything you would to write the hand-crafted XSLT plus how the mapper inserts your XSLT from the scripting functoid.
Left by Andy Brown on Aug 02, 2004 6:57 PM

# re: BizTalk Server 2004: Using the mapper for complex transforms - don't!!
Requesting Gravatar...
Charles,

I think I am in agreement with you here, not because I have hit the limits but from a more conceptual perspective. I think this also extends to developing schema too. The BTS tools are great to a point and then stop.

Enterprise/large scale transformations need to make use of all the features in XSLT. This includes the ability to factor reusable transformations into multiple files that are included etc. and thus reused across the company. XSLT development, debugging and so forth can be achieved with efficiency in tools such as XMLSpy or your choice.

Again, with schema, if you are are in the enterprise world with large (such as government) publicly owned and defined schema the BTS schema editor is a tad noddy but it does get you past first base.

MS tools are generally not enterprise focused but more productivity focused. Building web services is a great example. Build a class, put some attributes on it and hey presto a web service - or build an orchestration and create a web service. This code first approach is rapid in the short term or for small projects. Ultimately you need to design the messaging and then XSD and then WSDL and then build the code implementing the interface. To do this you need good XSD and WSDL tools (like notepad or XMLSpy).

Left by James Saull on Aug 03, 2004 1:28 PM

# re: BizTalk Server 2004: Using the mapper for complex transforms - don't!!
Requesting Gravatar...
To Alan: Simple when you know how! Create a new BizTalk map and add your source and destination schemas in the normal way. Then click on the grid to bring up your grid properties. Set the 'Custom XSLT Path' property to point to your bespoke XSLT file.

To Andy: Thanks for that. I'm aware from the general techie chatter within my company (which focuses on BizTalk integration) that many of my colleagues use the approach you outline all the time. As I said in the blog, I haven't had occassion to do anything very 'serious' with the mapper before, so I look forward to experimenting further. I guess the trick is to work out up front the best way of utilising the mapper's strengths and avoiding its weaknesses.

To James: Hi James :-) Hey...I know you!

I've just spent a couple of days with a two developers new to BizTalk, and found myself stressing to them repeatedly the need to really engage with XSLT, XSD and all things XML. I agree with you. Microsoft are pretty good at productivity tools, but these are always constrained in what they can do (or do well). Again, it's a matter of making good jusgments about how to get the best for these tools while avoiding their shortcomings.

Left by Charles Young on Aug 04, 2004 9:47 AM

# re: BizTalk Server 2004: Using the mapper for complex transforms - don't!!
Requesting Gravatar...
Thanks Charles for raising this issue. For the last two iterations I have had issues with the Biztalk mapper and so by 2004 I will definitely not use it. This tool has failed me in the past for delivering my solution on time because I wasted too much time conforming to it. Biztalk mapper promotes 'RAD' but unfortunately I find it more time intensive.

Left by Stevend on Aug 16, 2004 5:45 AM

# re: BizTalk Server 2004: Using the mapper for complex transforms - don't!!
Requesting Gravatar...
<xsl:template name="MyXsltConcatTemplate">
<xsl:param name="param1" />
<xsl:param name="param2" />
<xsl:element name="{$param1}">
<xsl:value-of select="$param2" />
</xsl:element>
</xsl:template>
Left by Ramkumar on Oct 12, 2004 11:51 AM

# re: BizTalk Server 2004: Using the mapper for complex transforms - don't!!
Requesting Gravatar...
Hi Charles,

Good post. I too do a great deal of work in XSLT, specifically grouping, searching, sorting, using the key feature in XSLT are all not supported by the mapper.

I do use the mapper to get started however, because I often find it tedious to type the namespace specifiers and the local name specifiers. So I create a new map with the schemas I want, then do a few of the simple "drag straight across" type connections, validate the map and save the XSLT off in it's own file.

Then I load it up in XML Spy and complete the task using the XSLT debugger in XML Spy. You can also use the XPath expression navigator in XML Spy to make your life with complex XPath expression more enjoyable.

Cheers,

-CP
Left by Curt Peterson on Oct 15, 2004 12:44 AM

# re: BizTalk Server 2004: Using the mapper for complex transforms - don't!!
Requesting Gravatar...
Hi..

I need to transform HIPAA compliant EDI transactions such as 837(Claims) into the Trizetto Facets Keyword format. The structure of the keyword file is totally different from that of the input EDI file.

The mappings are not always direct between the fields; there's conditional logic in many places.

This transformation is possible using ECMap, but i would like to know how feasible it is using Biztalk Mapper.

If you have used ECMap and if you have worked with Trizetto HIPAA gateway, you will have a better idea of what i am talking about.

Please let me know your suggestions.

Thanks,
Jagan
Left by Jagan on Jan 31, 2005 6:47 AM

# re: BizTalk Server 2004: Using the mapper for complex transforms - don't!!
Requesting Gravatar...
Quick Question guys, i have a receive ftp location which looks for a file on an ftp site which activates the orchestration. I want the orchestration to grab data from sql server table, mix the data with the data from the flatfile gotten on the ftp and then insert into another sql table. I dont know enough about biztlak to know where to start. I tried to create a paralell process which has two receive shapes which are connected to FTP Receive Loc and SqlServer respectively. Now It seems that i need to combine two messages into one and then try to map this 3rd composite message into a 4th one thats going to be sent to an SqlServer port? Do i need to create a schema which can hold both files separate under a root node and then MAP that schema into final result? Pretty confusing. By the way, i've used paralell process because it said that Two Receive Shapes in a convoy require them to have the same port....pretty weird. What's a good way to procede? Thanks
Left by Cell on Dec 05, 2005 3:20 AM

# re: BizTalk Server 2004: Using the mapper for complex transforms - don't!!
Requesting Gravatar...
Charles,

Can we use C# script in custom XSLT ?. Gimme a sample pls..

Left by Selvan on Dec 14, 2005 7:19 AM

# re: BizTalk Server 2004: Using the mapper for complex transforms - don't!!
Requesting Gravatar...
One problem I found with using the "Custom Xslt Path" capability in the mapper is that it always converts the path to a relative path. Also, in order to change the location of the XSLT file, you need to recompile. This can be annoying if you need to place your XSLT files in a different location on production. Is there a way to work around this issue? For now I've just created a custom helper to perform XSLT transformations.
Left by Waldemar on Jan 18, 2006 5:01 PM

# re: BizTalk Server 2004: Using the mapper for complex transforms - don't!!
Requesting Gravatar...
Jagan,
Do you have schemas that define the keyword format?
Eric
Left by Eric Stott on May 04, 2006 2:49 PM

# re: BizTalk Server 2004: Using the mapper for complex transforms - don't!!
Requesting Gravatar...
Hi I am using Table looping functoids in my mapper since for 1 record in incoming xml I need to generate 2 nodes on output but there is a problem wherever I dont mention the data it gives a blank element which I dont want is there a way to handle that
Left by Rajeev on Mar 22, 2007 3:24 PM

# re: BizTalk Server 2004: Using the mapper for complex transforms - don't!!
Requesting Gravatar...
jagan,
I too am looking at using BT 2009 to replace our Trizetto HIPAA gateway and would love to stay in touch re: ECMap and other issues as I look at the feasability of an entire replacement.
Left by Bryan on May 07, 2009 6:58 PM

# re: BizTalk Server 2004: Using the mapper for complex transforms - don't!!
Requesting Gravatar...
Another nifty case for using a custom XSLT in lieu of using the GUI:
suppose you have an xml doc that you want to give to users as a nicely renderd report that they can open in any browser, but to do so one must include a reference to an xsl document in the xml report file that must exist somewhere and that said users must be able to access at all times...
solution - use that xsl in the map and output just the rendered report as an html file!
-works great
Left by Will on Oct 02, 2009 11:23 AM

Your comment:
 (will show your gravatar)


Copyright © Charles Young | Powered by: GeeksWithBlogs.net