Welcome!

Industrial IoT Authors: Pat Romanski, Liz McMillan, Elizabeth White, Jeev Trika, Scott Allen

Related Topics: Industrial IoT

Industrial IoT: Article

Open Source Database Special Feature: An Introduction to Berkeley DB XML

Basic concepts, the shell commands, and beyond

In this article I am going to introduce you to the latest version of the Berkeley DB XML, version 2.2.8. Berkeley DB XML (BDB XML) is built on top of the well-known Berkeley Database (BDB). BDB XML is an open source, native XML database. Like its ancestor, BDB, it's an embedded database. It provides APIs for the Java, C++, Perl, Python, PHP, and Tcl languages. It supports the popular XML query languages XQuery and XPath 2.0. I will show you how to use BDB XML in two ways. This month I will introduce the BDB XML shell, and next month we will explore using BDB XML with Java. BDB XML has a lot of features, and I will try to cover the most important ones.

What's an Embedded Database?
Some of you may be familiar with embedded databases. An embedded database runs within another program. It is not a stand-alone server such as Oracle, DB2, or eXist. It is the programmer's responsibility to invoke the proper API for the database. For example, although Berkeley DB XML has transaction support, in order to activate it, API calls have to be made by the programmer. Many popular relational and native XML databases run in client-server mode. Oracle, DB2, and eXist run as client-server applications, while database server runs as a stand-alone application and many clients can connect to it. Clients and server programs run in different address spaces, probably even on different machines. A connection can be established between them using the JDBC, or ODBC protocols. In an embedded database such as BDB XML, this is not the case; clients and server both run in the same address space, and they are actually the same program. The client manages the Berkeley DB XML through API calls. Next month I will give some Java examples, which will clarify what an embedded database is.

Installation
Installing BDB XML on the Windows operating system is very easy. BDB XML comes with an installer for Windows. If you are using a UNIX-like operating system you have to build BDB XML from the source code. See "Building Berkeley DB XML for UNIX/POSIX systems" for detailed build instructions. I installed BDB XML on my laptop, a 1.4GHz Pentium M processor running on Windows XP Professional with 512Mb main memory, without any problems. The installer will automatically set the necessary Path and Classpath environment variables.

Using dbxml Shell
The shell is located under the bin directory of the BDB XML installation directory. It should be in your Path, so you can run it from any location at the Windows command line.

Invoke the shell by typing "dbxml," and type "quit" to terminate the shell at any time. In order to list all available commands, type "help." To obtain detailed information about a command, type help and the command name. For example "help createContainer" will display usage information of the createContainer command.

createContainer
First of all we have to create a container in which to store our data. A container in BDB XML is similar to a folder. Many XML files can reside in a container. In this article I am going to use XBench sample XML files and queries. You can download the XML sample files and the XML Schema from www.cs.umb.edu/~smimarog/xmlsample/.

Let's create a container called xbench.dbxml. You can, of course, name it anything you want.

dbxml> createContainer xbench.dbxml
Creating node storage container with nodes indexed

The user can choose between two storage types: Wholedoc or Node. The default is Node type storage. If you choose Wholedoc storage, the XML files will be stored as they are with all of the white space preserved. Node storage performs better. The BDB XML documentation suggests not using Wholedoc storage for documents bigger than 1MB.

putDocument
Now we are ready to put some XML data into xbench.dbxml container. In BDB XML the opened container is the default container. We have to open xbench.dbxml before working with it.

dbxml> openContainer xbench.dbxml

Actually, dbxml shell commands on Windows are case insensitive, so it's possible to write opencontainer instead of openContainer, but I am going to follow the established naming conventions here.

dbxml> putDocument sample_10 C:\dictionary10.xml f
Document added, name = sample_10

The putDocument command takes three arguments: the first argument is the unique identifier of the document, the second argument is file name, and the third is either f or s. I chose the name sample_10 as the unique identification of this XML document. C:\dictionary10.xml is the file name, including the path. Finally, the last argument f states that it's a file. Instead of providing a filename it's possible to provide XML data itself within quotes. There are several examples in the "Introduction to Berkeley DB XML" document, which uses an XML string instead of a file name.

Query
As mentioned earlier, BDB XML supports XQuery and XPath 2.0 languages. There will be several examples for each language (see Listings 1-4). BDB XML found 28 results for this query. Note that in order to retrieve the results we have to use "print" command.

setVerbose
BDB XML will produce more explanation about what it does when the setVerbose feature is turned on. This command takes two numeric arguments: level and category. Using this command, let's get more information regarding the XQuery Example2 (see Listing 5).

No indices are used in this query. The execution time is around 48.5 seconds. Execution time is very high for this query for two reasons: this is not a trivial query, and there are no indices created yet. I will conceptually explain the possible execution plan for this query:

For each $entry as /dictionary/e
  For each descendant $desc of $entry
   For each value $val of $desc
   Check if 'hockey' is a substring
    of $val
   return $entry/hwg hw

There are three for loops in this algorithm. At the heart of the three loops there is a very costly substring (contains) operation. Considering that no indices have been created, execution time is pretty good. Creating indices improves the query performance dramatically (see Listings 6-7). (You can find more information about indices in the Indexing section of this article.)

More Stories By Selim Mimaroglu

Selim Mimaroglu is a PhD candidate in computer science at the University of Massachusetts in Boston. He holds an MS in computer science from that school and has a BS in electrical engineering.

Comments (5) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
SYS-CON Belgium News Desk 12/20/05 01:07:02 PM EST

Open Source Database Special Feature: An Introduction to Berkeley DB XML. In this article I am going to introduce you to the latest version of the Berkeley DB XML, version 2.2.8. Berkeley DB XML (BDB XML) is built on top of the well-known Berkeley Database (BDB). BDB XML is an open source, native XML database. Like its ancestor, BDB, it's an embedded database. It provides APIs for the Java, C++, Perl, Python, PHP, and Tcl languages. It supports the popular XML query languages XQuery and XPath 2.0. I will show you how to use BDB XML in two ways. This month I will introduce the BDB XML shell, and next month we will explore using BDB XML with Java. BDB XML has a lot of features, and I will try to cover the most important ones.

SYS-CON Canada News Desk 12/20/05 12:23:13 PM EST

Open Source Database Special Feature: An Introduction to Berkeley DB XML. In this article I am going to introduce you to the latest version of the Berkeley DB XML, version 2.2.8. Berkeley DB XML (BDB XML) is built on top of the well-known Berkeley Database (BDB). BDB XML is an open source, native XML database. Like its ancestor, BDB, it's an embedded database. It provides APIs for the Java, C++, Perl, Python, PHP, and Tcl languages. It supports the popular XML query languages XQuery and XPath 2.0. I will show you how to use BDB XML in two ways. This month I will introduce the BDB XML shell, and next month we will explore using BDB XML with Java. BDB XML has a lot of features, and I will try to cover the most important ones.

SYS-CON Germany News Desk 12/20/05 11:39:03 AM EST

Open Source Database Special Feature: An Introduction to Berkeley DB XML. In this article I am going to introduce you to the latest version of the Berkeley DB XML, version 2.2.8. Berkeley DB XML (BDB XML) is built on top of the well-known Berkeley Database (BDB). BDB XML is an open source, native XML database. Like its ancestor, BDB, it's an embedded database. It provides APIs for the Java, C++, Perl, Python, PHP, and Tcl languages. It supports the popular XML query languages XQuery and XPath 2.0. I will show you how to use BDB XML in two ways. This month I will introduce the BDB XML shell, and next month we will explore using BDB XML with Java. BDB XML has a lot of features, and I will try to cover the most important ones.

SYS-CON UK News Desk 12/20/05 10:48:56 AM EST

Open Source Database Special Feature: An Introduction to Berkeley DB XML. In this article I am going to introduce you to the latest version of the Berkeley DB XML, version 2.2.8. Berkeley DB XML (BDB XML) is built on top of the well-known Berkeley Database (BDB). BDB XML is an open source, native XML database. Like its ancestor, BDB, it's an embedded database. It provides APIs for the Java, C++, Perl, Python, PHP, and Tcl languages. It supports the popular XML query languages XQuery and XPath 2.0. I will show you how to use BDB XML in two ways. This month I will introduce the BDB XML shell, and next month we will explore using BDB XML with Java. BDB XML has a lot of features, and I will try to cover the most important ones.

XML News Desk 12/20/05 10:12:48 AM EST

Open Source Database Special Feature: An Introduction to Berkeley DB XML. In this article I am going to introduce you to the latest version of the Berkeley DB XML, version 2.2.8. Berkeley DB XML (BDB XML) is built on top of the well-known Berkeley Database (BDB). BDB XML is an open source, native XML database. Like its ancestor, BDB, it's an embedded database. It provides APIs for the Java, C++, Perl, Python, PHP, and Tcl languages. It supports the popular XML query languages XQuery and XPath 2.0. I will show you how to use BDB XML in two ways. This month I will introduce the BDB XML shell, and next month we will explore using BDB XML with Java. BDB XML has a lot of features, and I will try to cover the most important ones.

@ThingsExpo Stories
There will be new vendors providing applications, middleware, and connected devices to support the thriving IoT ecosystem. This essentially means that electronic device manufacturers will also be in the software business. Many will be new to building embedded software or robust software. This creates an increased importance on software quality, particularly within the Industrial Internet of Things where business-critical applications are becoming dependent on products controlled by software. Qua...
In addition to all the benefits, IoT is also bringing new kind of customer experience challenges - cars that unlock themselves, thermostats turning houses into saunas and baby video monitors broadcasting over the internet. This list can only increase because while IoT services should be intuitive and simple to use, the delivery ecosystem is a myriad of potential problems as IoT explodes complexity. So finding a performance issue is like finding the proverbial needle in the haystack.
Machine Learning helps make complex systems more efficient. By applying advanced Machine Learning techniques such as Cognitive Fingerprinting, wind project operators can utilize these tools to learn from collected data, detect regular patterns, and optimize their own operations. In his session at 18th Cloud Expo, Stuart Gillen, Director of Business Development at SparkCognition, discussed how research has demonstrated the value of Machine Learning in delivering next generation analytics to imp...
The 19th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Digital Transformation, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportuni...
The Internet of Things will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform. In his session at @ThingsExpo, Craig Sproule, CEO of Metavine, demonstrated how to move beyond today's coding paradigm and shared the must-have mindsets for removing complexity from the develo...
SYS-CON Events announced today that MangoApps will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. MangoApps provides modern company intranets and team collaboration software, allowing workers to stay connected and productive from anywhere in the world and from any device.
Large scale deployments present unique planning challenges, system commissioning hurdles between IT and OT and demand careful system hand-off orchestration. In his session at @ThingsExpo, Jeff Smith, Senior Director and a founding member of Incenergy, will discuss some of the key tactics to ensure delivery success based on his experience of the last two years deploying Industrial IoT systems across four continents.
Basho Technologies has announced the latest release of Basho Riak TS, version 1.3. Riak TS is an enterprise-grade NoSQL database optimized for Internet of Things (IoT). The open source version enables developers to download the software for free and use it in production as well as make contributions to the code and develop applications around Riak TS. Enhancements to Riak TS make it quick, easy and cost-effective to spin up an instance to test new ideas and build IoT applications. In addition to...
Identity is in everything and customers are looking to their providers to ensure the security of their identities, transactions and data. With the increased reliance on cloud-based services, service providers must build security and trust into their offerings, adding value to customers and improving the user experience. Making identity, security and privacy easy for customers provides a unique advantage over the competition.
"We've discovered that after shows 80% if leads that people get, 80% of the conversations end up on the show floor, meaning people forget about it, people forget who they talk to, people forget that there are actual business opportunities to be had here so we try to help out and keep the conversations going," explained Jeff Mesnik, Founder and President of ContentMX, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
"There's a growing demand from users for things to be faster. When you think about all the transactions or interactions users will have with your product and everything that is between those transactions and interactions - what drives us at Catchpoint Systems is the idea to measure that and to analyze it," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York Ci...
Internet of @ThingsExpo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 19th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world and ThingsExpo Silicon Valley Call for Papers is now open.
I wanted to gather all of my Internet of Things (IOT) blogs into a single blog (that I could later use with my University of San Francisco (USF) Big Data “MBA” course). However as I started to pull these blogs together, I realized that my IOT discussion lacked a vision; it lacked an end point towards which an organization could drive their IOT envisioning, proof of value, app dev, data engineering and data science efforts. And I think that the IOT end point is really quite simple…
"My role is working with customers, helping them go through this digital transformation. I spend a lot of time talking to banks, big industries, manufacturers working through how they are integrating and transforming their IT platforms and moving them forward," explained William Morrish, General Manager Product Sales at Interoute, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
You think you know what’s in your data. But do you? Most organizations are now aware of the business intelligence represented by their data. Data science stands to take this to a level you never thought of – literally. The techniques of data science, when used with the capabilities of Big Data technologies, can make connections you had not yet imagined, helping you discover new insights and ask new questions of your data. In his session at @ThingsExpo, Sarbjit Sarkaria, data science team lead ...
Extracting business value from Internet of Things (IoT) data doesn’t happen overnight. There are several requirements that must be satisfied, including IoT device enablement, data analysis, real-time detection of complex events and automated orchestration of actions. Unfortunately, too many companies fall short in achieving their business goals by implementing incomplete solutions or not focusing on tangible use cases. In his general session at @ThingsExpo, Dave McCarthy, Director of Products...
WebRTC is bringing significant change to the communications landscape that will bridge the worlds of web and telephony, making the Internet the new standard for communications. Cloud9 took the road less traveled and used WebRTC to create a downloadable enterprise-grade communications platform that is changing the communication dynamic in the financial sector. In his session at @ThingsExpo, Leo Papadopoulos, CTO of Cloud9, discussed the importance of WebRTC and how it enables companies to focus...
Verizon Communications Inc. (NYSE, Nasdaq: VZ) and Yahoo! Inc. (Nasdaq: YHOO) have entered into a definitive agreement under which Verizon will acquire Yahoo's operating business for approximately $4.83 billion in cash, subject to customary closing adjustments. Yahoo informs, connects and entertains a global audience of more than 1 billion monthly active users** -- including 600 million monthly active mobile users*** through its search, communications and digital content products. Yahoo also co...
A critical component of any IoT project is what to do with all the data being generated. This data needs to be captured, processed, structured, and stored in a way to facilitate different kinds of queries. Traditional data warehouse and analytical systems are mature technologies that can be used to handle certain kinds of queries, but they are not always well suited to many problems, particularly when there is a need for real-time insights.
Amazon has gradually rolled out parts of its IoT offerings in the last year, but these are just the tip of the iceberg. In addition to optimizing their back-end AWS offerings, Amazon is laying the ground work to be a major force in IoT – especially in the connected home and office. Amazon is extending its reach by building on its dominant Cloud IoT platform, its Dash Button strategy, recently announced Replenishment Services, the Echo/Alexa voice recognition control platform, the 6-7 strategic...