Industrial IoT Authors: Liz McMillan, Elizabeth White, Pat Romanski, Kevin Jackson, Scott Allen

Related Topics: Industrial IoT

Industrial IoT: Article

Processing XML with C# and .NET

A solution that's simpler than you might expect

Microsoft's counterpiece to Java, the new C# programming language with its rich .NET library, uses XML as a core technology. This article presents some basic ideas, for example creating and manipulating a DOM tree, and reading and writing XML streams.

I also compare .NET's solution with the SAX model, and finally I show how a complex XSLT algorithm can be more simply implemented in C#. The source code can be downloaded from and www.sys-con.com/xml/sourcec.cfm.

XML Processors
To process an XML document means to extract information from it. Often the extracted information should be output to a new XML (or perhaps HTML) document that's similar to the original one - then we address the transformation.

Which processor to use for a given task is not a trivial question. There are a number of ready-to-use XML processors on the market - like Cocoon and Axkit - and any XML-enabled browser (like Netscape Navigator 6 or Microsoft Internet Explorer 6.0) has a built-in XML processor. They require processing instructions, which can be contained in an XSLT document for complex operations. XSLT is, however, a special style and not a very convenient programming language. Many would prefer to fiddle around with well-known, conventional programming structures from C++ or Java. For them, a real alternative is to write their own XML processor. Modern programming languages like C# and Java offer great support for this task in the form of class libraries; in the case of C#, these are .NET classes.

Processing Methods
There are two ways to process an XML document: online and offline. Offline processing means not being connected to the XML source, so the document has to be loaded (typically as a DOM tree) into memory beforehand. In the space-time trade, you lose space (memory) but gain time (speed) for the processing. This is the best method if most parts of the document are going to be processed, especially if they're going to be processed repeatedly.

Online processing (reading the document piece by piece) can be slow, but it works with less memory. It is advantageous if only certain parts of the document will be processed, or if the processing is sequentially straightforward.

C#'s built-in .NET library provides all the classes necessary for processing an XML document both ways. They are placed in the System.Xml namespace (with its nested namespaces System.Xml.Schema, System.Xml.Xsl, etc.).

The abstract classes XmlReader/XmlWriter provide the basis for online processing; XmlReader represents a fast, read-only forward cursor in an XML stream, while XmlWriter provides an interface for producing XML streams. The basis for offline processing is the class System.Xml.XmlDocument (with its superclass XmlNode) representing an XML document as a DOM tree (i.e., memory intensive). We'll investigate this option first.

Creating an XML Document
The XmlDocument constructor (usually called without parameters) creates an empty in-memory XML document. Its methods, like InsertBefore, InsertAfter, and AppendChild, build the DOM tree in memory. They need XmlNode parameters, which can be created by the XmlDocument's method CreateXxx, where Xxx stands for Element, Attribute, Node, Comment, ProcessingInstruction, CDataSection, DocumentFragment, XmlDeclaration, DocumentType, or EntityReference. They create a node in the context of the document but don't attach it to the tree:

XmlDocument doc = new XmlDocument();
// empty
XmlElement elem = doc.CreateElement
("NewAndOnlyElement"); // XmlElement
is a special XmlNode
elem.InnerText = "Data of the element";
... // many other properties to
configure the element
// appends node to document
// save the document to a file

The content of the text file data.xml (after putting this program segment into a Main method and running it) is:

<NewAndOnlyElement>Data of the element

In this way an XML tree of any complexity can be built step by step:

XmlNode node = doc.CreateComment
("This is a comment"); // XmlComment
is subclass of XmlNode
node = doc.CreateProcessingInstruction
("xml-stylesheet", // target
"type='text/xsl' href=data.xsl");
// data
node = doc.CreateCDataSection("<p>");

Since XmlDocument's AppendChild method is inherited from XmlNode, nodes can be appended not only to doc but also to any of the created nodes.

Navigating in the DOM Tree
After building (or loading) an XML tree, you can navigate over it and manipulate it with XmlDocument's properties (most of them are inherited from XmlNode). They have intelligible names:

  • DocumentElement delivers the root XmlElement of the tree.
  • Indexer [ ] delivers the XmlElement with the specified index.
  • Attributes delivers an XmlAttributeCollection object, a collection of XmlAttribute (a special Xml-Node) objects.
  • ParentNode, PreviousSibling, NextSibling, FirstChild, and LastChild navigate over the neighboring XmlNode objects for further manipulation or evaluation.
  • ChildNodes delivers all nodes as XmlNodeList, a collection of Xml-Node objects - it can be read, for example, with foreach.
  • NamespaceURI, Prefix, BaseURI, LocalName, and (qualified) Name deliver string objects about the document's URI.
  • OuterXml delivers a string containing the markup (XML text) of the node (with all its children).
  • InnerXml gets or sets a string containing the markup of all the children.
  • InnerText gets or sets a string, the concatenated text values (without XML markups) of the node and all its children.
  • PreserveWhitespace (with set), HasChildNodes, and IsReadOnly are bool properties.

Reading XML Data
There are additional ways to fill an XmlDocument object with data. The most important is its Load method with a string or stream, or TextReader or XmlReader parameter. It assumes that the XML document exists in text form in a stream, e.g., in a text file. The string-Parameter can be any URI. The difference between the overloaded versions is the navigation over the document before loading (because there is no need to load the whole document, only parts of it). Stream or Text-Reader allows sequential navigation; XmlReader can navigate to any component of the XML tree before reading it (see online processing in the next section).

If we have the XML data as a string object, we can to use the LoadXML method:

const string xmlData =
"<family location='Orlando' class='middle'>" +
"<lastname>Solymosi</lastname>" +
"<father>Andrew</father>" +
doc.Load(new StringReader(xmlData)); // equivalent alternative

Most void methods of XmlDocument (many inherited from XmlNode) suite to read and write the content of the DOM tree to and from a stream.

  • Load (with parameter Stream or string or TextReader or XmlReader) and LoadXml(string) fill an XmlDocument object with data.
  • Save (with parameter Stream or string or TextWriter or XmlWriter) writes the content of an XmlDocument object to a stream.
  • WriteTo writes OuterXML (markup of the root node); Write-Content-To writes InnerXML (the markup of all children) to an XmlWriter stream.
  • Normalize puts the tree into a "normal" form in which only markup separates XmlText nodes (i.e., there are no adjacent XmlText nodes).
  • RemoveChild and RemoveAll delete child element(s).

Online Processing
The abstract classes XmlReader and XmlWriter define an infrastructure for online processing, i.e., without building the DOM tree in memory. XmlReader is, however, not an implementation of the SAX model, rather a compromise between DOM (with the simple programming interface) and SAX (non-cached forward-only reader allowing the user to skip data). While a SAX reader activates the application's callback methods (push model), the methods of an XmlReader class must be called by the application (pull model). Some of the advantages of the latter are:

  • SAX requires the user to build complex state machines. A client for XmlReader builds a top-down procedural refinement.
  • XmlReader allows the client to read multiple streams (e.g., for aggregation); with SAX it's hard work.
  • An XmlReader application can be built on top of the SAX model (layering).
  • SAX copies the data from the parser buffer into the client's string object; XmlReader can use the client's string parameter as parser buffer so a string copy can be avoided.
  • SAX calls the client for each item (including attributes, processing instructions, and white space), while XmlReader's client can skip items (selective processing).

XmlReader has three implementations in .NET 1.1: XmlTextReader (this fast reader checks only well formedness and throws XmlException on failure), XmlValidatingReader (additionally validates against a DTD or a schema), and XmlNodeReader (for reading a DOM subtree). XmlWriter has only one implementation in .NET 1.1: XmlTextWriter. The counterpiece to XmlNodeReader (the class XmlNodeWriter) has not yet been implemented. However, people are welcome to write their own custom XmlReader and XmlWriter, extending the standard's functionality.

The following method shows how XmlReader moves along the document stream and displays the name of each element:

static void ReadMyDocument
(XmlReader reader) {
while (reader.Read()) {
if (reader.NodeType ==
XmlNodeType.Element ||
reader.NodeType ==
Console.Write (
(reader.NodeType ==
XmlNodeType.EndElement ?
"/" : "") + reader.Name + " ");

This method can be called with an XmlReader implementation as argument, e.g., with XmlText-Reader:

public static void Main
(String[] args) {
XmlReader r = new
XmlTextReader(args[0]); // URL

If the command-line parameter contained the name of a text file with the content

<family location='Orlando'

the output would be

family lastname /lastname father /father /family

This shows how XmlReader sequentially goes through the input document. A stop can be made at a node, which can be inspected with XmlReader's methods and properties:

string qName = reader.Name;
string localName = reader.LocalName;
string namespaceURI =
XmlNodeType nodeType =
string value = reader.Value;
bool hasAttributes =
int numberOfAttributes =

XmlReader properties (all read-only, i.e., without set) deliver information about the current node. The most important are:

  • Eof, HasAttributes, HasValue, IsDefault, IsEmptyElement deliver bool.
  • AttributeCount and Depth deliver int.
  • BaseURI, indexer [ ], LocalName, Name, NamespaceURI, Prefix, Value and XmlLang deliver string.
  • NodeType delivers an XmlNodeType enumeration value: one of Document, Text, Xml-Declaration, Element, EndElement, Entity, EndEntity, EntityReference, Attribute, Comment, Processing-Instruction, CDATA, DocumentFragment, DocumentType, Notation, White-space, SignificantWhitespace, or None. It can be used in a switch.

XmlReader's void methods navigate forward alongside the document:

  • Read delivers the bool value true if the next node is read successfully and throws XmlException if it detects an error in well formedness.
  • ReadXxx reads the next required node (otherwise throws XmlException) with Xxx = AttributeValue (with parameter Text, EntityReference or EndEntity) ElementString (simple text-only element), StartElement, EndElement, InnerXml, OuterXml, or String (contents of an element or text node).
  • MoveToXxx (with Xxx = Content, Element, Attribute, FirstAttribute or Next Attribute) moves to the required node and skips everything else.

The ReadXxx and MoveXxx methods perform a depth-first traversal of the tree, i.e., in the order that the sequential document stores it. The classic method for processing a document is to read it in a while cycle and switch according to XmlNodeType (see Listing 1).

As you can see from Listing 1, XmlReader requires knowledge about the current node type before calling the appropriate method, and SAX calls the appropriate method, and according to the current node type.

XmlNode provides methods for navigating the DOM tree horizontally and vertically to the neighboring nodes (like ChildNodes, FirstChild, LastChild, ParentNode, NextSibling, and PreviousSibling). Jumping to an arbitrary node requires an XPath expression. XmlNode's methods SelectXxx can be used to find a single node or all nodes that match a given criteria. The method SelectSingleNode returns the first node matching the search, from the top of the tree down, in document order. The method Select- Nodes returns an XmlNodeList object, a container for XmlNode objects that can be read by a foreach statement:

XmlNode root = doc.DocumentElement;
XmlNodeList nodeList =
root.SelectNodes("..."); // XPath
expression foreach (XmlNode node in
nodeList) {
... // work with the single nodes

The XmlNodeList object contains references to the underlying document's nodes. By modifying the value of a node, that node is also updated in the document and vice versa.

For more sophisticated navigation (e.g., over any XML-enabled data store like a database, or also for XSLT), the .NET namespace System.Xml.XPath contains a number of classes and interfaces. Its core is the abstract class XPathNavigator defining the common functionality of all navigators. The classes XmlNode (because it implements the interface IXPathNavigable) and XmlDocument export the method CreateNavigator for creating an XPathNavigator object:

XmlDocument doc = new XmlDocument();
XmlElement root = doc.DocumentElement;
// or any other node
XPathNavigator navigator =
Console.WriteLine("" +

It's interesting that XPathNavigator is an abstract class with no public subclass in .NET. CreateNavigator still delivers an object; its type is unknown to the user. The last line reveals it: System.Xml.DocumentXPathNavigator, a class not published in .NET. It might be an inner type of the class XmlNode, or perhaps its publication has been forgotten.

This object supports general navigation methods: selecting nodes, iterating over the selection, copying, moving, removing, and so on. The methods of XPathNavigator accept an XPath expression in the form of a string or a precompiled expression object; they are evaluated to identify the matching set of nodes.

The class XPathDocument is an optimized version of XmlDocument for XSLT processing and XPath queries; it provides a read-only, high-performance cache.

Writing XML
XmlWriter is an abstract class (like XmlReader) defining the base functionality for producing an XML document. The concept of .NET's XmlWriter is very similar to SAX. XmlTextWriter is currently its only implementation of the .NET class library; XmlNodeWriter (the counterpiece to XmlNodeReader) is hopefully coming in a future release. They work just like the reader versions but in the opposite directions.

XmlTextWriter provides the WriteXxx methods for writing out typed elements and attributes, where Xxx = StartDocument, EndDocument, DocType, StartElement, EndElement, FullEndElement, ElementString, StartAttribute, EndAttribute, Attributes, AttributeString, Comment, ProcessingInstruction, String, Base64, BinHex, CData, CharEntity, Chars, EntityRef, Name, NmToken, Node, QualifiedName, Raw, SurrogateCharEntity, or Whitespace. It's possible, for example, to write out the whole current node in an XmlReader (with or without attributes) by calling WriteNode:

static void Serialize(XmlReader
reader, XmlWriter writer) {
while (reader.Read())
writer.WriteNode(reader, true); // true = with default attributes

XmlTextWriter supports different output stream types (its constructor takes a string [file or URI], a Stream, or a TextWriter) and is configurable. Its properties specify whether to provide namespace support (bool property Namespace), indentation options (int property Intendation, char property IndentChar), quote character for attribute values (char property QuoteCahr), lexical representation for typed values, and so on. Listing 2 illustrates how some of these properties can be configured:

XSL Transformations
The namespace System.Xml.Xsl contains the class XslTransform, which manages XSLT transformations. It uses XPathNavigator during the transformation process. XslTransform reads as input an XML document, an XSLT document, and some optional parameters (of type XsltArgumentList). It can produce any text-based output.

To perform a transformation, an XslTransform object must be loaded with the XSLT document (as a DOM tree): the Load method requires a string (URL) parameter. Next an XPathDocument object (instead of XmlDocument - it offers a better performance for XSLT processing) must be created and initialized. Finally, the Transform method executes the transformation:

static void TransformXML(string xml, // name of file to transform
string xslt, // file name of stylesheet or XSLT document
XmlTextWriter writer) {
XslTransform transformator = new XslTransform();
transformator.Load(xslt); // stylesheet loaded
XPathDocument document = new XPathDocument(xml); // DOM tree created
transformator.Transform(document, null, writer, null);

The last call of Transform could take some additional XSLT parameters (XsltArgumentList and XmlResolver) - here they are null. Replacing XSLT Functions by C# Most transformations can be elegantly solved in XSLT. Nevertheless there are some tasks that need special tricks.

Let's assume an XML document with a root element <matrix> contains a number of (let's say n) <row> elements and all of them contains the same number n of <column> elements with the text 1_1, 1_2, ..., 1_n, 2_1, 2_2, ..., 2_n, ..., n_1, n_2, ... n_n:

1_1   1_2   ...   1_n
2_1   2_2   ...   2_n
n_1   n_2   ...   n_n

The task is to exchange the row and column elements (to mirror the table along its diameter):

1_1   2_1   ...   n_1
1_2   2_2   ...   n_2
1_n   2_n   ...   n_n

This transformation is called transposing a matrix. Because XSLT is a functional language with no variables, this task can be solved only recursively (see Listing 3).

The template recursive is called here inside the template recursive; this recursion implements a for-cycle of a procedural programming language (like C#) that doesn't exist in XSLT (for-each can iterate only over the children of a node, it's not the for-cycle of C# or Java!).

The same algorithm can be expressed in C# as shown in Listing 4 (Listings 4-6 are available at www.sys-con.com/xml/sourcec.cfm).

It's remarkable that this C# program doesn't use variables; the different values (between 1 and n) of the (constant) parameter COLUMN have been stored on the stack by the repeated calls of the method Recursive. So recursion has been "misused" for a variable - in XSLT (as in any other functional language) this is the only way to solve the problem. If we use C# variables, recursion is evitable (see Listing 5).

Here the counter variable column takes the values 1 through n instead of the parameter COLUMN in the XSLT program.

The opulence of the .NET library and the object-oriented nature of C# allows another, even simpler solution of hanging over the InnerText objects in the references of the original XmlDocument (i.e., offline) (see Listing 6).

.  .  .

As you can see, sometimes a self-programmed XSLT processor can solve a problem in a much simpler manner than a prefabricated one.

Comments (2) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

Most Recent Comments
tom berry 09/18/07 09:54:59 AM EDT

get rid of the annoying pop-up ads if you want more people to visit this site.

@ThingsExpo Stories
"delaPlex is a software development company. We do team-based outsourcing development," explained Mark Rivers, COO and Co-founder of delaPlex Software, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
IoT is rapidly changing the way enterprises are using data to improve business decision-making. In order to derive business value, organizations must unlock insights from the data gathered and then act on these. In their session at @ThingsExpo, Eric Hoffman, Vice President at EastBanc Technologies, and Peter Shashkin, Head of Development Department at EastBanc Technologies, discussed how one organization leveraged IoT, cloud technology and data analysis to improve customer experiences and effi...
The IoT is changing the way enterprises conduct business. In his session at @ThingsExpo, Eric Hoffman, Vice President at EastBanc Technologies, discussed how businesses can gain an edge over competitors by empowering consumers to take control through IoT. He cited examples such as a Washington, D.C.-based sports club that leveraged IoT and the cloud to develop a comprehensive booking system. He also highlighted how IoT can revitalize and restore outdated business models, making them profitable ...
There are several IoTs: the Industrial Internet, Consumer Wearables, Wearables and Healthcare, Supply Chains, and the movement toward Smart Grids, Cities, Regions, and Nations. There are competing communications standards every step of the way, a bewildering array of sensors and devices, and an entire world of competing data analytics platforms. To some this appears to be chaos. In this power panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, Bradley Holt, Developer Advocate a...
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, wh...
Machine Learning helps make complex systems more efficient. By applying advanced Machine Learning techniques such as Cognitive Fingerprinting, wind project operators can utilize these tools to learn from collected data, detect regular patterns, and optimize their own operations. In his session at 18th Cloud Expo, Stuart Gillen, Director of Business Development at SparkCognition, discussed how research has demonstrated the value of Machine Learning in delivering next generation analytics to imp...
The cloud market growth today is largely in public clouds. While there is a lot of spend in IT departments in virtualization, these aren’t yet translating into a true “cloud” experience within the enterprise. What is stopping the growth of the “private cloud” market? In his general session at 18th Cloud Expo, Nara Rajagopalan, CEO of Accelerite, explored the challenges in deploying, managing, and getting adoption for a private cloud within an enterprise. What are the key differences between wh...
Connected devices and the industrial internet are growing exponentially every year with Cisco expecting 50 billion devices to be in operation by 2020. In this period of growth, location-based insights are becoming invaluable to many businesses as they adopt new connected technologies. Knowing when and where these devices connect from is critical for a number of scenarios in supply chain management, disaster management, emergency response, M2M, location marketing and more. In his session at @Th...
Basho Technologies has announced the latest release of Basho Riak TS, version 1.3. Riak TS is an enterprise-grade NoSQL database optimized for Internet of Things (IoT). The open source version enables developers to download the software for free and use it in production as well as make contributions to the code and develop applications around Riak TS. Enhancements to Riak TS make it quick, easy and cost-effective to spin up an instance to test new ideas and build IoT applications. In addition to...
When people aren’t talking about VMs and containers, they’re talking about serverless architecture. Serverless is about no maintenance. It means you are not worried about low-level infrastructural and operational details. An event-driven serverless platform is a great use case for IoT. In his session at @ThingsExpo, Animesh Singh, an STSM and Lead for IBM Cloud Platform and Infrastructure, will detail how to build a distributed serverless, polyglot, microservices framework using open source tec...
Apixio Inc. has raised $19.3 million in Series D venture capital funding led by SSM Partners with participation from First Analysis, Bain Capital Ventures and Apixio’s largest angel investor. Apixio will dedicate the proceeds toward advancing and scaling products powered by its cognitive computing platform, further enabling insights for optimal patient care. The Series D funding comes as Apixio experiences strong momentum and increasing demand for its HCC Profiler solution, which mines unstruc...
IoT offers a value of almost $4 trillion to the manufacturing industry through platforms that can improve margins, optimize operations & drive high performance work teams. By using IoT technologies as a foundation, manufacturing customers are integrating worker safety with manufacturing systems, driving deep collaboration and utilizing analytics to exponentially increased per-unit margins. However, as Benoit Lheureux, the VP for Research at Gartner points out, “IoT project implementers often ...
It is one thing to build single industrial IoT applications, but what will it take to build the Smart Cities and truly society changing applications of the future? The technology won’t be the problem, it will be the number of parties that need to work together and be aligned in their motivation to succeed. In his Day 2 Keynote at @ThingsExpo, Henrik Kenani Dahlgren, Portfolio Marketing Manager at Ericsson, discussed how to plan to cooperate, partner, and form lasting all-star teams to change t...
In his general session at 18th Cloud Expo, Lee Atchison, Principal Cloud Architect and Advocate at New Relic, discussed cloud as a ‘better data center’ and how it adds new capacity (faster) and improves application availability (redundancy). The cloud is a ‘Dynamic Tool for Dynamic Apps’ and resource allocation is an integral part of your application architecture, so use only the resources you need and allocate /de-allocate resources on the fly.
In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, provided an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life sett...
Presidio has received the 2015 EMC Partner Services Quality Award from EMC Corporation for achieving outstanding service excellence and customer satisfaction as measured by the EMC Partner Services Quality (PSQ) program. Presidio was also honored as the 2015 EMC Americas Marketing Excellence Partner of the Year and 2015 Mid-Market East Partner of the Year. The EMC PSQ program is a project-specific survey program designed for partners with Service Partner designations to solicit customer feedbac...
Cloud computing is being adopted in one form or another by 94% of enterprises today. Tens of billions of new devices are being connected to The Internet of Things. And Big Data is driving this bus. An exponential increase is expected in the amount of information being processed, managed, analyzed, and acted upon by enterprise IT. This amazing is not part of some distant future - it is happening today. One report shows a 650% increase in enterprise data by 2020. Other estimates are even higher....
SYS-CON Events announced today that Bsquare has been named “Silver Sponsor” of SYS-CON's @ThingsExpo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. For more than two decades, Bsquare has helped its customers extract business value from a broad array of physical assets by making them intelligent, connecting them, and using the data they generate to optimize business processes.
19th Cloud Expo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterpri...
Internet of @ThingsExpo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 19th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devices - comp...