jul 21 anders jeppesen base64 blog Microsoft CRM Microsoft CRM 2011 PI / XI SAP SAP PI SAP XI Scribe unicode

Scribe Integration using unicode and MSMQ queues


The issue with Scribe and unicode is right now that it’s not supported using the standard engine.
That means that for example Russian and Polish letters will be auto converted into ANSI which in many cases becomes nonsense….but wait…..continue reading and you will get the solution on to solve this and still being able to sleep in the nights because when Scribe will support this in the next version you will just remove the wrapper and use the standard engine.
In respect for Scribesoft and all the hard work they are putting in the new version of Scribe I have found a solution where we still are using the standard Scribe engine and then just having a wrapper around the system doing the Unicode magic.
I have tried to illustrate it in the diagram below.
So this means that when Scribe is ready with the next version the wrapper would easily be removed without redesigning the complete DTS.


MSMQ Unicode to BASE64 encoding service from Alfapeople
Microsoft CRM 2011 plugin BASE64 to Unicode from Alfapeople
It’s not possible for me to provide the source code for the public, as Alfapeople have the copyright, but you are more than welcome to contact me if you have a business case related to this, then we will figure out a solution that will work for you and your customer.
If you need to contact me related to this please write to me on:
                             anj@alfapeople.com or anders@anjep.dk

Description of MSMQ Unicode encoding service from AlfaPeople

This is a windows service that is running on a given server, I would suggest the scribe server as it is not using any resources.
In a nutshell it’s taking the xml message and converts all values into BASE64 without doing anything to the schema itself.
It’s very simple in the configuration and it’s all done from a configuration file.
          Multi queues IN and OUT.
          Multi-threading with option to choose how many
          Blacklisting (If there is element you don’t want to encode, and there is as numbers don’t need encoding)
          Option to choose the query interval
          Option to choose the encoding direction from or to BASE64 encoding
          Special feature to validate data format inside text strings
          Error handling where messages goes into a special queue

Description of Microsoft CRM 2011 plugin BASE64 to Unicode from AlfaPeople

This is a normal plugin who is converting all values who is validated as BASE64 encoded into Unicode.
It’s only triggered if a special integration field has a date value in that way we are able to control the conversion as we have cases where we are moving data directly into CRM as the don’t contains any special letters.
For some scenarios we have also made blacklist functionality here as some of the blacklisted field from the service can have the look of a BASE64 encoding, so to be able to ensure that we don’t get any double conversion we have this functionality.


There is many way of handling this issue and it all depends of the situation in what direction you should go, but with the scenario above you have a real time integration with only one extra loop on the path.
When you have understood the concept it’s no problem working with this in the solution and also in heavy loaded environments it doesn’t seems to slow down the progress as the web service from Microsoft still is the bottleneck.
Let me know if you have any questions to this and I will be more than happy to try helping you out
Share on FacebookTweet about this on TwitterShare on LinkedInShare on Google+

4 comments on “Scribe Integration using unicode and MSMQ queues

Skriv et svar

Din e-mailadresse vil ikke blive offentliggjort. Krævede felter er markeret med *