Welcome!

Industrial IoT Authors: Pat Romanski, William Schmarzo, Elizabeth White, Stackify Blog, Yeshim Deniz

Related Topics: Industrial IoT

Industrial IoT: Article

Processing XML with C# and .NET

A solution that's simpler than you might expect

Microsoft's counterpiece to Java, the new C# programming language with its rich .NET library, uses XML as a core technology. This article presents some basic ideas, for example creating and manipulating a DOM tree, and reading and writing XML streams.

I also compare .NET's solution with the SAX model, and finally I show how a complex XSLT algorithm can be more simply implemented in C#. The source code can be downloaded from and www.sys-con.com/xml/sourcec.cfm.

XML Processors
To process an XML document means to extract information from it. Often the extracted information should be output to a new XML (or perhaps HTML) document that's similar to the original one - then we address the transformation.

Which processor to use for a given task is not a trivial question. There are a number of ready-to-use XML processors on the market - like Cocoon and Axkit - and any XML-enabled browser (like Netscape Navigator 6 or Microsoft Internet Explorer 6.0) has a built-in XML processor. They require processing instructions, which can be contained in an XSLT document for complex operations. XSLT is, however, a special style and not a very convenient programming language. Many would prefer to fiddle around with well-known, conventional programming structures from C++ or Java. For them, a real alternative is to write their own XML processor. Modern programming languages like C# and Java offer great support for this task in the form of class libraries; in the case of C#, these are .NET classes.

Processing Methods
There are two ways to process an XML document: online and offline. Offline processing means not being connected to the XML source, so the document has to be loaded (typically as a DOM tree) into memory beforehand. In the space-time trade, you lose space (memory) but gain time (speed) for the processing. This is the best method if most parts of the document are going to be processed, especially if they're going to be processed repeatedly.

Online processing (reading the document piece by piece) can be slow, but it works with less memory. It is advantageous if only certain parts of the document will be processed, or if the processing is sequentially straightforward.

C#'s built-in .NET library provides all the classes necessary for processing an XML document both ways. They are placed in the System.Xml namespace (with its nested namespaces System.Xml.Schema, System.Xml.Xsl, etc.).

The abstract classes XmlReader/XmlWriter provide the basis for online processing; XmlReader represents a fast, read-only forward cursor in an XML stream, while XmlWriter provides an interface for producing XML streams. The basis for offline processing is the class System.Xml.XmlDocument (with its superclass XmlNode) representing an XML document as a DOM tree (i.e., memory intensive). We'll investigate this option first.

Creating an XML Document
The XmlDocument constructor (usually called without parameters) creates an empty in-memory XML document. Its methods, like InsertBefore, InsertAfter, and AppendChild, build the DOM tree in memory. They need XmlNode parameters, which can be created by the XmlDocument's method CreateXxx, where Xxx stands for Element, Attribute, Node, Comment, ProcessingInstruction, CDataSection, DocumentFragment, XmlDeclaration, DocumentType, or EntityReference. They create a node in the context of the document but don't attach it to the tree:

XmlDocument doc = new XmlDocument();
// empty
XmlElement elem = doc.CreateElement
("NewAndOnlyElement"); // XmlElement
is a special XmlNode
elem.InnerText = "Data of the element";
... // many other properties to
configure the element
doc.AppendChild(elem);
// appends node to document
doc.Save("data.xml");
// save the document to a file

The content of the text file data.xml (after putting this program segment into a Main method and running it) is:

<NewAndOnlyElement>Data of the element
</NewAndOnlyElement>

In this way an XML tree of any complexity can be built step by step:

XmlNode node = doc.CreateComment
("This is a comment"); // XmlComment
is subclass of XmlNode
doc.AppendChild(node);
node = doc.CreateProcessingInstruction
("xml-stylesheet", // target
"type='text/xsl' href=data.xsl");
// data
doc.AppendChild(node);
node = doc.CreateCDataSection("<p>");
...

Since XmlDocument's AppendChild method is inherited from XmlNode, nodes can be appended not only to doc but also to any of the created nodes.

Navigating in the DOM Tree
After building (or loading) an XML tree, you can navigate over it and manipulate it with XmlDocument's properties (most of them are inherited from XmlNode). They have intelligible names:

  • DocumentElement delivers the root XmlElement of the tree.
  • Indexer [ ] delivers the XmlElement with the specified index.
  • Attributes delivers an XmlAttributeCollection object, a collection of XmlAttribute (a special Xml-Node) objects.
  • ParentNode, PreviousSibling, NextSibling, FirstChild, and LastChild navigate over the neighboring XmlNode objects for further manipulation or evaluation.
  • ChildNodes delivers all nodes as XmlNodeList, a collection of Xml-Node objects - it can be read, for example, with foreach.
  • NamespaceURI, Prefix, BaseURI, LocalName, and (qualified) Name deliver string objects about the document's URI.
  • OuterXml delivers a string containing the markup (XML text) of the node (with all its children).
  • InnerXml gets or sets a string containing the markup of all the children.
  • InnerText gets or sets a string, the concatenated text values (without XML markups) of the node and all its children.
  • PreserveWhitespace (with set), HasChildNodes, and IsReadOnly are bool properties.

Reading XML Data
There are additional ways to fill an XmlDocument object with data. The most important is its Load method with a string or stream, or TextReader or XmlReader parameter. It assumes that the XML document exists in text form in a stream, e.g., in a text file. The string-Parameter can be any URI. The difference between the overloaded versions is the navigation over the document before loading (because there is no need to load the whole document, only parts of it). Stream or Text-Reader allows sequential navigation; XmlReader can navigate to any component of the XML tree before reading it (see online processing in the next section).

If we have the XML data as a string object, we can to use the LoadXML method:

const string xmlData =
"<family location='Orlando' class='middle'>" +
"<lastname>Solymosi</lastname>" +
"<father>Andrew</father>" +
"</family>";
doc.LoadXml(xmlData);
doc.Load(new StringReader(xmlData)); // equivalent alternative

Most void methods of XmlDocument (many inherited from XmlNode) suite to read and write the content of the DOM tree to and from a stream.

  • Load (with parameter Stream or string or TextReader or XmlReader) and LoadXml(string) fill an XmlDocument object with data.
  • Save (with parameter Stream or string or TextWriter or XmlWriter) writes the content of an XmlDocument object to a stream.
  • WriteTo writes OuterXML (markup of the root node); Write-Content-To writes InnerXML (the markup of all children) to an XmlWriter stream.
  • Normalize puts the tree into a "normal" form in which only markup separates XmlText nodes (i.e., there are no adjacent XmlText nodes).
  • RemoveChild and RemoveAll delete child element(s).

Online Processing
The abstract classes XmlReader and XmlWriter define an infrastructure for online processing, i.e., without building the DOM tree in memory. XmlReader is, however, not an implementation of the SAX model, rather a compromise between DOM (with the simple programming interface) and SAX (non-cached forward-only reader allowing the user to skip data). While a SAX reader activates the application's callback methods (push model), the methods of an XmlReader class must be called by the application (pull model). Some of the advantages of the latter are:

  • SAX requires the user to build complex state machines. A client for XmlReader builds a top-down procedural refinement.
  • XmlReader allows the client to read multiple streams (e.g., for aggregation); with SAX it's hard work.
  • An XmlReader application can be built on top of the SAX model (layering).
  • SAX copies the data from the parser buffer into the client's string object; XmlReader can use the client's string parameter as parser buffer so a string copy can be avoided.
  • SAX calls the client for each item (including attributes, processing instructions, and white space), while XmlReader's client can skip items (selective processing).

XmlReader has three implementations in .NET 1.1: XmlTextReader (this fast reader checks only well formedness and throws XmlException on failure), XmlValidatingReader (additionally validates against a DTD or a schema), and XmlNodeReader (for reading a DOM subtree). XmlWriter has only one implementation in .NET 1.1: XmlTextWriter. The counterpiece to XmlNodeReader (the class XmlNodeWriter) has not yet been implemented. However, people are welcome to write their own custom XmlReader and XmlWriter, extending the standard's functionality.

The following method shows how XmlReader moves along the document stream and displays the name of each element:

static void ReadMyDocument
(XmlReader reader) {
while (reader.Read()) {
if (reader.NodeType ==
XmlNodeType.Element ||
reader.NodeType ==
XmlNodeType.EndElement)
Console.Write (
(reader.NodeType ==
XmlNodeType.EndElement ?
"/" : "") + reader.Name + " ");
}
}

This method can be called with an XmlReader implementation as argument, e.g., with XmlText-Reader:

public static void Main
(String[] args) {
XmlReader r = new
XmlTextReader(args[0]); // URL
ReadMyDocument(r);
}

If the command-line parameter contained the name of a text file with the content

<family location='Orlando'
class='middle'>
<lastname>Solymosi</lastname>
<father>Andrew</father>
</family>

the output would be

family lastname /lastname father /father /family

This shows how XmlReader sequentially goes through the input document. A stop can be made at a node, which can be inspected with XmlReader's methods and properties:

string qName = reader.Name;
string localName = reader.LocalName;
string namespaceURI =
reader.NamespaceURI;
XmlNodeType nodeType =
reader.NodeType;
string value = reader.Value;
bool hasAttributes =
reader.HasAttributes;
int numberOfAttributes =
reader.AttributeCount;
...

XmlReader properties (all read-only, i.e., without set) deliver information about the current node. The most important are:

  • Eof, HasAttributes, HasValue, IsDefault, IsEmptyElement deliver bool.
  • AttributeCount and Depth deliver int.
  • BaseURI, indexer [ ], LocalName, Name, NamespaceURI, Prefix, Value and XmlLang deliver string.
  • NodeType delivers an XmlNodeType enumeration value: one of Document, Text, Xml-Declaration, Element, EndElement, Entity, EndEntity, EntityReference, Attribute, Comment, Processing-Instruction, CDATA, DocumentFragment, DocumentType, Notation, White-space, SignificantWhitespace, or None. It can be used in a switch.

XmlReader's void methods navigate forward alongside the document:

  • Read delivers the bool value true if the next node is read successfully and throws XmlException if it detects an error in well formedness.
  • ReadXxx reads the next required node (otherwise throws XmlException) with Xxx = AttributeValue (with parameter Text, EntityReference or EndEntity) ElementString (simple text-only element), StartElement, EndElement, InnerXml, OuterXml, or String (contents of an element or text node).
  • MoveToXxx (with Xxx = Content, Element, Attribute, FirstAttribute or Next Attribute) moves to the required node and skips everything else.

The ReadXxx and MoveXxx methods perform a depth-first traversal of the tree, i.e., in the order that the sequential document stores it. The classic method for processing a document is to read it in a while cycle and switch according to XmlNodeType (see Listing 1).

As you can see from Listing 1, XmlReader requires knowledge about the current node type before calling the appropriate method, and SAX calls the appropriate method, and according to the current node type.

Navigation
XmlNode provides methods for navigating the DOM tree horizontally and vertically to the neighboring nodes (like ChildNodes, FirstChild, LastChild, ParentNode, NextSibling, and PreviousSibling). Jumping to an arbitrary node requires an XPath expression. XmlNode's methods SelectXxx can be used to find a single node or all nodes that match a given criteria. The method SelectSingleNode returns the first node matching the search, from the top of the tree down, in document order. The method Select- Nodes returns an XmlNodeList object, a container for XmlNode objects that can be read by a foreach statement:

XmlNode root = doc.DocumentElement;
XmlNodeList nodeList =
root.SelectNodes("..."); // XPath
expression foreach (XmlNode node in
nodeList) {
... // work with the single nodes
}

The XmlNodeList object contains references to the underlying document's nodes. By modifying the value of a node, that node is also updated in the document and vice versa.

For more sophisticated navigation (e.g., over any XML-enabled data store like a database, or also for XSLT), the .NET namespace System.Xml.XPath contains a number of classes and interfaces. Its core is the abstract class XPathNavigator defining the common functionality of all navigators. The classes XmlNode (because it implements the interface IXPathNavigable) and XmlDocument export the method CreateNavigator for creating an XPathNavigator object:

XmlDocument doc = new XmlDocument();
doc.Load("data.xml");
XmlElement root = doc.DocumentElement;
// or any other node
XPathNavigator navigator =
root.CreateNavigator();
Console.WriteLine("" +
navigator.GetType());

It's interesting that XPathNavigator is an abstract class with no public subclass in .NET. CreateNavigator still delivers an object; its type is unknown to the user. The last line reveals it: System.Xml.DocumentXPathNavigator, a class not published in .NET. It might be an inner type of the class XmlNode, or perhaps its publication has been forgotten.

This object supports general navigation methods: selecting nodes, iterating over the selection, copying, moving, removing, and so on. The methods of XPathNavigator accept an XPath expression in the form of a string or a precompiled expression object; they are evaluated to identify the matching set of nodes.

The class XPathDocument is an optimized version of XmlDocument for XSLT processing and XPath queries; it provides a read-only, high-performance cache.

Writing XML
XmlWriter is an abstract class (like XmlReader) defining the base functionality for producing an XML document. The concept of .NET's XmlWriter is very similar to SAX. XmlTextWriter is currently its only implementation of the .NET class library; XmlNodeWriter (the counterpiece to XmlNodeReader) is hopefully coming in a future release. They work just like the reader versions but in the opposite directions.

XmlTextWriter provides the WriteXxx methods for writing out typed elements and attributes, where Xxx = StartDocument, EndDocument, DocType, StartElement, EndElement, FullEndElement, ElementString, StartAttribute, EndAttribute, Attributes, AttributeString, Comment, ProcessingInstruction, String, Base64, BinHex, CData, CharEntity, Chars, EntityRef, Name, NmToken, Node, QualifiedName, Raw, SurrogateCharEntity, or Whitespace. It's possible, for example, to write out the whole current node in an XmlReader (with or without attributes) by calling WriteNode:

static void Serialize(XmlReader
reader, XmlWriter writer) {
while (reader.Read())
writer.WriteNode(reader, true); // true = with default attributes
}

XmlTextWriter supports different output stream types (its constructor takes a string [file or URI], a Stream, or a TextWriter) and is configurable. Its properties specify whether to provide namespace support (bool property Namespace), indentation options (int property Intendation, char property IndentChar), quote character for attribute values (char property QuoteCahr), lexical representation for typed values, and so on. Listing 2 illustrates how some of these properties can be configured:

XSL Transformations
The namespace System.Xml.Xsl contains the class XslTransform, which manages XSLT transformations. It uses XPathNavigator during the transformation process. XslTransform reads as input an XML document, an XSLT document, and some optional parameters (of type XsltArgumentList). It can produce any text-based output.

To perform a transformation, an XslTransform object must be loaded with the XSLT document (as a DOM tree): the Load method requires a string (URL) parameter. Next an XPathDocument object (instead of XmlDocument - it offers a better performance for XSLT processing) must be created and initialized. Finally, the Transform method executes the transformation:

static void TransformXML(string xml, // name of file to transform
string xslt, // file name of stylesheet or XSLT document
XmlTextWriter writer) {
XslTransform transformator = new XslTransform();
transformator.Load(xslt); // stylesheet loaded
XPathDocument document = new XPathDocument(xml); // DOM tree created
transformator.Transform(document, null, writer, null);
}

The last call of Transform could take some additional XSLT parameters (XsltArgumentList and XmlResolver) - here they are null. Replacing XSLT Functions by C# Most transformations can be elegantly solved in XSLT. Nevertheless there are some tasks that need special tricks.

Let's assume an XML document with a root element <matrix> contains a number of (let's say n) <row> elements and all of them contains the same number n of <column> elements with the text 1_1, 1_2, ..., 1_n, 2_1, 2_2, ..., 2_n, ..., n_1, n_2, ... n_n:

1_1   1_2   ...   1_n
2_1   2_2   ...   2_n
...
n_1   n_2   ...   n_n

The task is to exchange the row and column elements (to mirror the table along its diameter):

1_1   2_1   ...   n_1
1_2   2_2   ...   n_2
...
1_n   2_n   ...   n_n

This transformation is called transposing a matrix. Because XSLT is a functional language with no variables, this task can be solved only recursively (see Listing 3).

The template recursive is called here inside the template recursive; this recursion implements a for-cycle of a procedural programming language (like C#) that doesn't exist in XSLT (for-each can iterate only over the children of a node, it's not the for-cycle of C# or Java!).

The same algorithm can be expressed in C# as shown in Listing 4 (Listings 4-6 are available at www.sys-con.com/xml/sourcec.cfm).

It's remarkable that this C# program doesn't use variables; the different values (between 1 and n) of the (constant) parameter COLUMN have been stored on the stack by the repeated calls of the method Recursive. So recursion has been "misused" for a variable - in XSLT (as in any other functional language) this is the only way to solve the problem. If we use C# variables, recursion is evitable (see Listing 5).

Here the counter variable column takes the values 1 through n instead of the parameter COLUMN in the XSLT program.

The opulence of the .NET library and the object-oriented nature of C# allows another, even simpler solution of hanging over the InnerText objects in the references of the original XmlDocument (i.e., offline) (see Listing 6).

.  .  .

As you can see, sometimes a self-programmed XSLT processor can solve a problem in a much simpler manner than a prefabricated one.

Comments (2)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


IoT & Smart Cities Stories
Moroccanoil®, the global leader in oil-infused beauty, is thrilled to announce the NEW Moroccanoil Color Depositing Masks, a collection of dual-benefit hair masks that deposit pure pigments while providing the treatment benefits of a deep conditioning mask. The collection consists of seven curated shades for commitment-free, beautifully-colored hair that looks and feels healthy.
The textured-hair category is inarguably the hottest in the haircare space today. This has been driven by the proliferation of founder brands started by curly and coily consumers and savvy consumers who increasingly want products specifically for their texture type. This trend is underscored by the latest insights from NaturallyCurly's 2018 TextureTrends report, released today. According to the 2018 TextureTrends Report, more than 80 percent of women with curly and coily hair say they purcha...
The textured-hair category is inarguably the hottest in the haircare space today. This has been driven by the proliferation of founder brands started by curly and coily consumers and savvy consumers who increasingly want products specifically for their texture type. This trend is underscored by the latest insights from NaturallyCurly's 2018 TextureTrends report, released today. According to the 2018 TextureTrends Report, more than 80 percent of women with curly and coily hair say they purcha...
We all love the many benefits of natural plant oils, used as a deap treatment before shampooing, at home or at the beach, but is there an all-in-one solution for everyday intensive nutrition and modern styling?I am passionate about the benefits of natural extracts with tried-and-tested results, which I have used to develop my own brand (lemon for its acid ph, wheat germ for its fortifying action…). I wanted a product which combined caring and styling effects, and which could be used after shampo...
The platform combines the strengths of Singtel's extensive, intelligent network capabilities with Microsoft's cloud expertise to create a unique solution that sets new standards for IoT applications," said Mr Diomedes Kastanis, Head of IoT at Singtel. "Our solution provides speed, transparency and flexibility, paving the way for a more pervasive use of IoT to accelerate enterprises' digitalisation efforts. AI-powered intelligent connectivity over Microsoft Azure will be the fastest connected pat...
There are many examples of disruption in consumer space – Uber disrupting the cab industry, Airbnb disrupting the hospitality industry and so on; but have you wondered who is disrupting support and operations? AISERA helps make businesses and customers successful by offering consumer-like user experience for support and operations. We have built the world’s first AI-driven IT / HR / Cloud / Customer Support and Operations solution.
Codete accelerates their clients growth through technological expertise and experience. Codite team works with organizations to meet the challenges that digitalization presents. Their clients include digital start-ups as well as established enterprises in the IT industry. To stay competitive in a highly innovative IT industry, strong R&D departments and bold spin-off initiatives is a must. Codete Data Science and Software Architects teams help corporate clients to stay up to date with the mod...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
Druva is the global leader in Cloud Data Protection and Management, delivering the industry's first data management-as-a-service solution that aggregates data from endpoints, servers and cloud applications and leverages the public cloud to offer a single pane of glass to enable data protection, governance and intelligence-dramatically increasing the availability and visibility of business critical information, while reducing the risk, cost and complexity of managing and protecting it. Druva's...
BMC has unmatched experience in IT management, supporting 92 of the Forbes Global 100, and earning recognition as an ITSM Gartner Magic Quadrant Leader for five years running. Our solutions offer speed, agility, and efficiency to tackle business challenges in the areas of service management, automation, operations, and the mainframe.