For the name field the mapping is a little complex as it contains the subfields first name and last name. Thus it's a container as well as a contained object itself. The following control attributes should be added for it.

• t2xml:container="true": To tell if it's a container for first name and last name
• t2xml:object="true": To tell if it's also a contained object inside Employee
• t2xml:object_sep="os:spacechar": To tell the os-specific space character as delimiter
• t2xml:container_type="delimited": To tell the Container type as delimited
• t2xml:escape_char="?": To tell about the escape character

Fixed-length Flat File Case
For the fixed-length case too let's consider a record from the flat file.

777227878Simi? D Roy 123000.00

The fixed-length field case is almost the same as the delimited case; the only difference is that here the field lengths are fixed and not separated by any delimiters. Therefore, the control attributes for the Employee element are a little different from those of the delimited case.

• t2xml:container="true": Added to tell it's a container or record
• t2xml:container_type="fixed": Added to tell the record type is fixed
• t2xml:container_endtoken="os:linesep": Added to tell the os-specific line separator is used as a record terminator

For the contained objects there is an additional attribute to specify the object length. The respective lengths of the ssn, name, and salary fields were updated in the t2xml:object_len attribute. So, for contained objects here are the required attributes:

• t2xml:object="true": Added to tell it's a contained object
• t2xml:object_len="9","30","9": For ssn, name, and salary fields

The complete schema definition for the fixed-length case can be found in the file fixedLength-sample.xsd in the source jar.

Default Instance Generation
Based on the schema defined for the flat file data structure, a default XML instance that follows the rules defined in the schema definition is generated. Generating this XML instance from the schema definition requires the proper identification of the root element to start the instance generation. This XML instance generation from the schema is implemented in the xerces.xs.instance.SchemaInstance class. This class can generate an XML instance from any schema definition, provided the schema contains at least one element declaration. If there is more than one element declaration, it calls for the element with no reference from other elements as a potential candidate for the root element. A generic root element finder is implemented in the xerces.xs.instance.RootElementFinder class. The XML instance generation in the SchemaInstance class starts with the root element and comes up with XML elements on traversal of complextypes and elements defined in the root element definition. It converts each complex type/simple type element to a default XML instance node during schema traversal. If some elements are optional, it produces one default element. If some elements have maxOccurs="unbounded," it goes with the value defined in Instance.MAX_UNBOUND_OCCURANCE. The most important methods of SchemaInstance are shown in Listing 2.

The source jar contains the full class declaration for SchemaInstance class. All the handler methods in the class are recursively called to generate the XML instance. Below is the sample code that demonstrates how to use this class to generate an XML instance from a schema definition:

SchemaInstance lSchemaInstance= new SchemaInstance(aSchemaFilePathOrURL);
lSchemaInstance.generate(aOutputFilePath); /* or */
lSchemaInstance.generateInstance(aOutputStream);

The XML instance generated from the schema example above looks like this:

<root_element>
<Employee>
<ssn/>
<name>
<fname/>
<lname/>
</name>
<salary/>
</Employee>
</root_element>

Populating the Instance With Data
As populating the XML instance with data comes after the XML instance is generated, one point worth mentioning here is that the XML instance to be filled up with data does not go on with the whole schema definition at one time; rather, each schema element contained in the schema definition under the root element is converted to the XML element and then filled up with data. When the XML instance generation starts from the root element, the control attributes are examined for each schema element that is generated as XML.

These control attributes tell if an element is a container or contained object, and also tell about the container end token, object separator, etc. Therefore, depending upon the control attributes, after the XML instance is generated for a particular schema element, the physical record is read from the flat file and the instance is populated with live data from flat file. This process is repeated for each record defined in the flat file. Only after traversal of the full schema definition (starting from the root element) will a filled-up instance representing the full schema definition be created. If the maxOccurs attribute is "unbounded" for a schema element, the number of XML instances for this element is created as per the availability of records in the flat file; otherwise the actual number is regarded in the schema definition.

The lookup for the control attributes and their correct handling is very important when filling the XML instance with data. To start, the implementation class xerces.xs.instance.T2xmlInstance implements the t2xml.IParseProperties and extends the xerces.xs.instance.SchemaInstance class. In the SchemaInstance class, the bare-bones XML elements were generated, but in the derived T2xmlInstance, with the help of the IParseProperties, these XML instances will now be filled up with data from the flat file. In Listing 3 you can see a skeleton representation of the T2xmlInstance class.

The last three methods in the class skeleton were overridden in the T2xmlInstance class from the SchemaInstance class to fill up the XML elements generated in the SchemaInstance class.

The getRootSchemaElement(.) method is overridden to find the root element in accordance with the control attribute "t2xml:rootelem."

The handleParticle(.) method is overridden to look up specific control attributes for container and container type, so that the filler object that fills the data to the generated XML instance is set up properly.

The fillupData(.) method is overridden to fill up data in the XML instance based on the control attribute for object marking and the type of filler object passed in.

The other methods in the class are helper methods to get the instance fill-up mechanism working. The full source code for T2xmlInstance may be found in the source jar. The T2xmlInstance class uses the control attributes explained in Table 1 to populate the default XML instance with data. To run T2xmlInstance as an application, download source.jar, unzip the contents, and try the following commands for the delimited sample case and the fixed-length case, respectively. {your jdk home}\bin\java -cp .;classes;lib\xerces\resolver.jar;lib\xerces\xercesI mpl.jar;lib\xerces\xml-apis.jar;lib\xerces\xmlParserAPIs.jar xerces.xs.instance. T2xmlInstance test\delimited-sample.xsd test\delimited-input.txt test\delimited-output.xml

{your jdk home}\bin\java -cp .;classes;lib\xerces\resolver.jar;lib\xerces\xercesI
mpl.jar;lib\xerces\xml-apis.jar;lib\xerces\xmlParserAPIs.jar xerces.xs.instance.
T2xmlInstance test\fixedLength-sample.xsd test\fixedLength-input.txt test\fixedLength-output.xml

The XML instance for the delimited and fixed-length case is shown in Listing 4. Because the data structure is exactly the same for these two cases, the generated XML is also identical.

Scope for Future Enhancement
This article demonstrated the concept of converting flat files to an XML instance and provides a solution to prove the concept. Fairly complex flat files may be converted to XML instances using this approach, but the solution needs to be improved for industry-standard XML-instance generation because the support for doing custom calculations and mapping data directly to attributes is not yet established. Enabling XPath expressions might add some more goodies to the approach.

Conclusion
This approach is used to parse flat files and create an XML instance given a schema representation of the underlying data in the flat file. Many flat file?based systems such as EFT/EDI, custom database migration, and backup utilities will find this useful. Since this approach is based on open standards such as W3C schema and Xerces implementation for schema, it may be widely used.

References