Today's Dreaded Exception: Data at the root level is invalid.

Found a problem recently when serializing a custom object.  Here's what I was working with. Warning: this is the bathroom wall of code version. Do not copy/paste this into production.

public static XmlDocument SerializeToXmlDoc(Object obj) 
{ 
 try 
    { 
        XmlDocument xmlDoc = new XmlDocument(); 
        string xmlString =  SerializeIt(obj);             
        xmlDoc.LoadXml(xmlString); 
        return xmlDoc; 
    } 
    catch (Exception e) {   return null; } 
} 
 
public static string SerializeIt(Object obj) 
{ 
    try 
    {     
        MemoryStream memoryStream = new MemoryStream(); 
        XmlSerializer xs = new XmlSerializer(obj.GetType()); 
        XmlTextWriter xmlTextWriter = new XmlTextWriter(memoryStream, Encoding.UTF8); 
        xs.Serialize(xmlTextWriter, obj); 
 
        memoryStream = (MemoryStream)xmlTextWriter.BaseStream; 
 
        string xmlString = UTF8ByteArrayToString(memoryStream.ToArray()); 
        return xmlString; 
    } 
    catch (Exception e) { return null; } 
} 
 
private static string UTF8ByteArrayToString(byte[] characters) 
{ 
    UTF8Encoding encoding = new UTF8Encoding(); 
    String constructedString = encoding.GetString(characters); 
    return (constructedString); 
} 

The problem that was that as I passed one of my custom objects to it, I'd get this exception:
Data at the root level is invalid. Line 1, position 1.

Fine, root level… got it. Let's take a peek at what's actually trying to be loaded.

Wait, What The…?

Here's the kicker: there was a funny null/something character at the beginning of the string, and therefore, my XmlDoc couldn't successfully execute the LoadXml method. Confessional: I scraped this method from the interwebs and its bathroom wall of code. I tweaked it to suit my needs. I figured it was ready for ANY kind of POCO object. Guess not. I have hit the 10% case where it didn't work well. Well, let's fire up the Text Visualizer and figure out why that string isn't loading into an XmlDocument properly.


Text Visualizer Visual Studio xml string

Hmm. That's funny. What is that?! Turning to the Immediate Window didn't give any real answers as to the value of that unknown/bad character.

immediate window Visual Studio 2008
Hmm. Looks blank. I then copied that value right from the Immediate Window, pasted into Notepad, it comes out as a question mark. Nice! Here's the direct paste:

?xmlString.Substring(0,1)
"?"

Solved!

You could go down all kinds of kludgey roads and try to replace the first character if it's not angle bracket "<", or try and trim null chars, etc.

Turns out that a StringWriter will do the job well in this case. No more null character leading to a failed load of the string.
(via the bathroom wall of code at http://asp.net2.aspfaq.com/xml-serialization/simple-serialization.html)

public static string SerializeIt(Object obj) 
{ 
    try 
    {               
        XmlSerializer serializer = new XmlSerializer(obj.GetType()); 
        StringWriter sw = new StringWriter(); 
        serializer.Serialize(sw, obj); 
        sw.Close(); 
 
        // get the Xml as a string 
        string xmlString = sw.GetStringBuilder().ToString(); 
 
        return xmlString; 
    } 
    catch (Exception e) { return null; } 
}

 

 

 

The MemoryStream algorithm worked well for some of my POCO objects, but not for another.


Posted in: c# , visual studio , xml , serialization  Tags:
Actions: E-mail | Permalink | Comments (0) |
Phil posted on June 17, 2009 09:29

A new project has me writing up the same old Country/State/Province reference tables. My feeling is that these static (fairly static) entities should be normalized and referenced by foreign key. I had asked a StackOverflow question on whether other developers had this prebuilt set of country/state/province create scripts in their toolbelt.

This is a code garage sale! Never again worry about creating and loading country and province/state data.

Create The Schema

Create your Country and State tables with this CREATE SQL script. As always, name the State table whatever you like (ProvState, tblState, whathaveyou).  Some folks don't like table names to be the same as reserved keywords.

country_state_erd

Populating

Here is a collection of insert scripts to get the data populated quickly for you.

Have Fun

Some developers go further down the normalization path by creating a City table, but I usually pass on that. As always, that decision is largely dependant on the problem domain or task at hand.

Yippee for me – that’ll be fun trying to reconcile data when someone enters their city location incorrectly as “St. Paul” / “Tuscon” / “Pittsburg” instead of Saint Paul / Tucson/ Pittsburgh.

If you would like to contribute an insert script or two for a country that you would like to see here, just contact me with your script!


Posted in: code blowout , sql  Tags:
Actions: E-mail | Permalink | Comments (0) |
Phil posted on June 14, 2009 18:34

I'd like to share my favorite tech podcasts. These are in no particular order! You probably won't be surprised, it's the list of usual suspects circa 2008/2009. Click on each to go directly to the podcast's main page. You can find the RSS feed there!

Developer Centric Podcasts

.NET Rocks
itunes_dnr
Hanselminutes
itunes_hanselminutes
Thirsty Developer
ThirstyDeveloper
StackOverflow
stackoverflow
Herding Code
herdingcode
CodeCast
codecast
 

 

I.T. Industry or General Tech

RunAs Radio
itunes_runasradio
Windows Weekly
windowsweekly
This Week in Startups
twist
Search Engine
searchengine
This Week in Tech
twit
Security Now
securitynow
 

 

Scratched!

  • Alt.Net podcast - sorry guys, it was getting to sound a little too negative. I enjoyed the Domain Driven Design episode!
  • Polymorphic podcast - Craig sounds like a really nice guy, but there was a lull in the regularity of the podcasts, so I abandoned it.
  • ASP.NET podcast - didn't tickle my fancy. It's not you, it's me. Serious!

Wish List

  • Scott Gu -  Microsofties are blogging like crazy, I wonder if Scott has any time on his hands to do his own podcast. Ya right. He can be found as a guest here and there.

What great developer podcasts or I.T. podcasts are out there that are missing from this list?!? Are YOU starting a podcast and wanting some listeners? Let us all know, leave a comment!


Posted in: podcast , learning  Tags:
Actions: E-mail | Permalink | Comments (0) |
Phil posted on June 10, 2009 14:03

A previous position gave me the chance to learn about Lean Manufacturing. I've learned a small useful bit of insight about how a company, team or operation can should operate to deliver value for its customers. The Lean philosophy is all about identifying and eliminating waste. One of the biggest challenges is for companies to first IDENTIFY the waste in their organization and processes. The biggest target is the shop floor, but all areas of the organization should be heeding this warning. This isn't just a shop-floor solution.

What Is Waste, Anyway?

Waste is something that the customer is not, or does not want to pay for. Waste can typically be categorized as either ‘necessary waste’ and ‘pure waste’. The necessary waste can involve processes that are required to get the order taken, the accounting processes done, the payroll completed, and the reports compiled. The customer doesn’t pay for any of that, but it’s got to be done. Certainly there are processes that can be improved within those necessary waste items that you find during a value-stream mapping exercise. The pure waste is the main focus of a Lean organization’s focus.

Seven Forms of Waste

Overproduction - making too much product before it has been ordered, or before you can invoice for it. Simple stuff. Only product when and until you need it. Don't batch your product. A solution often found for this form of waste is single-piece-flow. There are a myriad of reasons excuses generated for why a company/process/team cannot do single piece flow. The waste here is the wasted opportunity where you could have been building something else that could have been invoiced right away.

Overprocessing  - only work on a product as much as it needs or as much as the customer wants. Do not polish the underside of a car. Do not give that widget 10 coats of paint if it only requires 2.

Inventory - typically this refers to raw materials and finished product. Having too much input and too many outputs not being shipped. Both of these tie up cash that should otherwise be sitting in your bank account. Produce the product at the last responsible time. Ship & invoice as soon as you can. This is a fine balance that can have you juggling with many variables (ordering lead times with multiple vendors, shipping schedules, lumpy customer demand, external dependencies, weather, etc.

Transportation - moving raw materials, in-process goods, and finished goods doesn't add value to the customer. They aren't paying for those trucks, forklifts, conveyors, and people - you are! This includes handoffs and transfers.

Waiting - imagine the assembly line. This is again the lost-opportunity scenario. Someone on the shop floor waiting for raw materials or their 'input' is a classic example. This applies to customers and vendors as well. Waiting for vendors who don't support you in the Lean philosophy can cost you time and money. Those vendors should be encouraged to accelerate/step up, or be replaced. We all have customers to serve, and we should strive to meet their demands.

Rework/Defects – don’t make something that isn't what the customer ordered. A defect may be a problem in your product that deviates from the specs, or is actually a hidden flaw. The later in the process that the flaws are discovered, the more expensive it will be to fix. This is totally non-value added, and you don't get paid to fix those defects. Rework is just as wasteful: fixing a processing mistake from an upstream process usually costs more than finding and fixing the problem upstream. Software developers and I.T. workers know this intimately.

Motion - the movement of people. If a worker has to travel outside a 6-foot circle (or some other subjective 'reasonable' distance), they likely are spending too much time not working on the product and adding value. This happens also when an often-needed set of files are located across the building. Same would apply to a printer, stockpile of supplies, or other team members!


Posted in: cost savings , customer , lean , waste  Tags:
Actions: E-mail | Permalink | Comments (0) |

When hammering away at a unit test, I often found myself wanting to test the current unit test and its associated code. I of course wanted to use the debugger, break points, inspect values, use the Watch tool, etc.

The First Iteration

  • Put the cursor in the test method.
  • Go to Visual Studio IDE’s menu 
  • Click Tools - Run - Tests in Current Context
  • Debug as per normal

ide

Uh, Do It Smarter

Then I remembered that you could put the shortcuts in the right click context menu. Ah ha!

rightclick

You can see where this is going if you paid attention to the first iteration, as well as the title of this blog post. Taking your hands off the keyboard to move to the mouse adds so much more wasted time in development. File this under “what were you thinking?”.

Smartest

Now it’s all keyboard goodness:

  • Cursor in the test method
  • use a cute mnemonic in your head when you're typing in this combo. Re Test
  • Ctrl-R, Ctrl-T

What is YOUR favorite Visual Studio keyboard shortcut?


Posted in: development , shortcuts , visual studio  Tags:
Actions: E-mail | Permalink | Comments (0) |
Phil posted on June 9, 2009 23:40

Like 95% of the searching masses, I use Google as the primary engine in my browser search fields. Sure, I have installed the usual suspects of other sites that I like to search by. OK, I only ever used Wikipedia once or twice, and Stack Overflow is nice to have. I find it difficult, to switch search engines WITHOUT using the mouse. Maybe it's just me, and there's a magical keyboard shortcut to switch to Next Engine, or something.

firefox_search_engines_list

 

Bing?!

After hearing the big news about MSN's rebrand, the launch of Microsoft's new search engine Bing on the usual podcasts, AND having Scott Hanselman challenge his readers to judge Bing after using for a month. I thought it would be a flash in the pan.

Loyalty and Affinity

Some people feel loyalty to brands, and some to search engines. Me, I just stick with Google because it works, and it looks clean. That's about it. I don't have any natural affinity to my search provider, so I am going to take the Bing Challenge. So at home, work and the laptop, I have changed my primary search providers to the 'blue b'.

 

You can see in the screenshot above that I really threw it a softball. Get ready Bing, people are paying attention!


Posted in: bing  Tags:
Actions: E-mail | Permalink | Comments (3) |
Phil posted on June 4, 2009 17:28

The Goal: Have your HL7 v3 converted by Mirth to a specified HL7 v2 message.

The Prerequisites

  • Using Mirth 1.8+. This may work with previous versions, but this solution hasn't been tested on anything but 1.8.
  • You know what your HL7 XML looks like. You have a sample message available.
  • You know what you want the HL7 v2 to look like.
  • You may be expecting repeating segments to be converted. That's OK, it will be covered here.

Let's Do It

1. Login to your Mirth Administrator, create a new channel.

  • Give your channel a name
  • Ensure the incoming datatype is set to HL7 v3
  • All the other defaults are OK
  • Save
1

2. Switch to the Source tab.

  • Ensure your Listener port is something unique.
  • All the other defaults are OK
1b

3. Switch to the Destinations tab.

  • Give the first Destination a good name
  • Change the connector type to anything
  • Save
3

4. Switch to the "Edit Transformer" menu option

  • Click "Add New Step"
  • Change the Type to "JavaScript". This walkthrough will take the JavaScript route as more code/mapping can be fit into one window. The other mappers may fit your style better!
  • Give it an appropriate name, hit Enter
4


4b

5. On the top right pane, click the Message Templates tab.

  • This is where your XML HL7 v3 template will go. If you don't have a template, you can make one up for your experimental/development purposes!
  • Find your XML HL7 v3 template, and paste it here.
55b

6. On the bottom right pane, find the Outbound Message Template. It will likely default to HL7 v3.0.

  • Ensure the data type dropdown shows HL7 v2.x.
  • Find your HL7 v2 template, and paste it here. Keep any default values that you have, but go through and prefix them with something unique to remind yourself to ensure that value is mapped from the HL7 v3 message. That'll help identify any mapping that has been missed.
66b

7. Click Message Trees

  • You'll see a tree representation of both your messages.
7
  • 8.     Now it's time to match up the elements that you want to go from the v3 to the v2. Draggy-droppy time! Repeat this for EACH data field that you want moved from the v3 to the v2.
  • Drill down into the Outbound Message Template. Find the v2 element that you want filled. (e.g. Patient Last Name at PID 5.1)
  • Pick the Green dot icon. Drag it over and drop onto to the JavaScript window.
  • You'll now have something like this in the JavaScript code window:
            tmp['PID']['PID.5']['PID.5.1']
  • Type the equals sign at the end of this line. We're going to assign this value to something after the next step. Now you'll have this:

tmp['PID']['PID.5']['PID.5.1'] =

  • Go back to the Inbound Message Template Tree. Drill down into where the Patient Last Name is at.
  • Drag and drop that Green dot icon over to the JavaScript window. Drop it off after the equals sign.
  • You should now have something like this code in your JavaScript window:
        tmp['PID']['PID.5']['PID.5.1']=msg['controlActProcess']['subject']['target']['identifiedPerson']['name']['family'].toString()   
  • Congrats, you've just mapped one property from v3 to v2.
  • Repeat the above step as necessary for all the fields that you need transformed from v3 to v2.
  • You code any steps by hand in JavaScript now that you have the basic syntax down. I'd suggest creating new Transformer steps for each v2 segment. It will help you find/fix problems that you find for a particular segment if/when they appear. It follows the idea of modularity.
88b
9. Use this sample channel for the full solution.  

10. Deploy your channel and use the Send Message command.

  • Copy/paste a V3 message with all the values filled in.
  • Click Send.
  • In the "Encoded Message" tab of the Destination Connector, you'll see your output HL7 v2.
 

Posted in: code blowout , mirth  Tags:
Actions: E-mail | Permalink | Comments (0) |

So your application needs to send emails to stakeholders/customers/admins/managers. You know the kind of emails:

  • password reminders,
  • account balances,
  • reminders & status updates
  • confirmations of all sorts

Typically my development routine consisted of something like this:

  1. Write a template email. String constants with placeholders, etc. Dear [FirstName] [LastName],
  2. Read all appropriate details from a data-store. You know, the things specific to the recipient of the email. Replace [FirstName] [LastName] with real names, or a link specific to their account to reset their password, etc.
  3. Write the email sending business logic. System.Net.Mail and all the goodness within. Attachments, BCC, SMTP, all those good things provided by that namespace. (Thank you Microsoft, for helping us forget CDONTS.)
  4. Put in your own email address to receive the unit tests and integration tests.
  5. Brace for the email flood into your mail client.
  6. Context switch between the mail client and development IDE. More back and forth for you!

New Tool in the Toolbox – Already Built into the .NET Framework!

While testing, you can have emails sent to a directory instead of being sent to SMTP server. Simply put this in your web.config:

<system.net> 
    <mailSettings> 
        <smtp deliveryMethod="SpecifiedPickupDirectory"> 
            <specifiedPickupDirectory pickupDirectoryLocation="c:\SomeEmailDirectory\" /> 
        </smtp> 
    </mailSettings> 
</system.net> 

 

So What?

So how much time does it really save you? It’s negligible, really, when your mail server is on the LAN. If you are working with an SMTP server that is unreliable, slow, or keeping accounting on mail count/charging for bandwidth, the seconds per sent email can add up.

No matter about SMTP servers… good, bad or ugly. You could monitor a directory with the .NET FileSystemWatcher to watch for newly dropped email files, and then open them automatically!

Imagine, your unit or integration tests create a new email to be sent, and another dev tool (using FileSystemWatcher) can automatically ShellExecute it for you. That saves you the time of switching to the directory, double clicking the newly created email, and then getting on with the work of checking for correctness.

Credit

Yep, this StackOverflow question keeps on giving!

http://stackoverflow.com/questions/54929/hidden-features-of-asp-net


Posted in: asp.net , c# , development , email , shortcuts , testing  Tags:
Actions: E-mail | Permalink | Comments (0) |