Welcome!

Industrial IoT Authors: Elizabeth White, Stackify Blog, Yeshim Deniz, SmartBear Blog, Liz McMillan

Related Topics: Java IoT, Industrial IoT, Weblogic, Eclipse, Machine Learning , Recurring Revenue

Java IoT: Article

Column Store, In-Memory, MPP Databases and Oracle

How Oracle implements latest developments in database world

( For latest information on Oracle 12c database update please refer to the following article: Oracle 12c Database and How It Relates to SAP Hana )

RDBMSs are stable and mature products. While there is nothing radically new on horizon that would challenge Codd's relational theory and related advances in data processing there are some developments that force established vendors like Oracle to come up with new features and products.

Column Stores and Oracle
Column store concept has been around for quite a while. Vendors like HP Vertica grabbed some market share in data warehousing segment with their column store,  MPP databases. Oracle Exadata is offering Hybrid Columnar Compression -  solution that modifies Oracle standard row based storage (NSM, or row-major) into proprietary format that is probably closer to PAX (as opposed to DSM) classification. Rows of data are reorganized, broken down into columns, compressed and stored in Compression Units which consist of multiple Oracle data blocks. CU is physically implemented as a standard Oracle single column chained row. (Description of CU layout is based on marvelous article written by Oracle expert  Jonathan Lewis).

This is truly hybrid design, i.e., column store is implemented on top of a standard row store.

It is not in the scope of this article to discuss pros and cons of this implementation from performance, locking, compression and other points of view. I will just mention that HCC requires change in standard operating procedures and methods.

MPP (shared nothing) and Oracle
Column-based stores like HP Vertica use Multiple Parallel Processing, shared nothing design to enhance performance by bringing processing closer to data, i.e., data is processed in parallel on the node where it resides. Volume of data that is moved around is reduced with the additional benefit of CPU and data proximity.

Oracle's implementation of this idea could be classified as asymmetric MPP. Oracle Exadata uses offloading to storage layer, Smart Scan, Storage Indexes and other techniques to improve performance.

Storage layer (Exadata cells) are tasked with as much work as possible to reduce load on database server and network. Each Exadata storage cell has the ability to perform some parts of data processing operations as well as decompression.

IMDB and Oracle
SAP relatively recently released Hana - a fully functional in-memory RDBMS, targeted for both OLTP and OLAP applications. Hana operates on the premise that whole database is in memory and not on disk. Majority of data processing is now pointer based arithmetic, so whole sections of RDBMS code related to moving data back and forth between disk, RAM and CPU are not needed any more. This is all possible because memory is more affordable and abundant, so much so that most of modern OLTP databases can completely fit within modern server's RAM.

Oracle puchased Times Ten in-memory database, but marketed it mostly as caching layer to standard Oracle database. Times Ten is not marketed as stand alone IMDB the way SAP Hana is.

Oracle database can have Flash Cache devices configured as an extension of SGA for better performance (via database parameter), or for database logging purposes.

Exadata can be configured with terabytes of Flash Cache memory for database caching and to serve as solid state disk. This is not memory directly accessible by CPU though ( DRAM ), i.e., Oracle database accesses Flash Cache via PCI interface and IO operating system calls. In other words, Flash is treated the same as disk, with all negative consequences of such approach regarding code complexity and performance. The latest release of Exadata performs writes directly to flash cache first to improve performance. We should expect more optimizations that will try to better utilize abundance of various types of memory. Expected scenario could be similar to Microsoft Hekaton project is also about adding IMDB features to SQL Server ( tables can be loaded in memory and processed in IMDB fashion, with reduced latching and locking; perhaps choice between different storage engines will be possible).

Conclusion
Oracle will probably continue to execute on strategy that worked well in the past - gradual inclusion of new technologies into its core RDBMS product (like it did with programmable server, OODBMS, Internet database, XML, partitioning, etc.). None of the latest developments in server technologies and database world is as seismic as introduction of RDBMS, client-server and Internet computing was. Oracle was so far successful  in modifying its database engine to adjust to the changes in hardware and data processing methods. We expect this trend to continue, as pace of technological innovation is somewhat slowing down and no truly disruptive changes are on horizon. Oracle solutions are designed to introduce and take advantage of these new (old) technologies and avoid cannibalizing existing profits. Intent is to maximally protect existing legacy RDBMS software revenues and integrate and sell products that came with new hardware and software acquisitions. Oracle Exadata, for example, could be viewed just as intelligent, Oracle database aware and Oracle produced SAN, bundled with database server. It is perhaps safe strategy in an environment where even mediocre and lackluster repackaging, modifying and integrating acquisitions is not seriously challenged by radical new ideas or strong, competitive implementations of existing concepts and technologies. SAP Hana, for example, is also unification layer built on top of in-house built or acquired products.

More Stories By Ranko Mosic

Ranko Mosic, BScEng, is specializing in Big Data/Data Architecture consulting services ( database/data architecture, machine learning ). His clients are in finance, retail, telecommunications industries. Ranko is welcoming inquiries about his availability for consulting engagements and can be reached at 408-757-0053 or [email protected]

IoT & Smart Cities Stories
Early Bird Registration Discount Expires on August 31, 2018 Conference Registration Link ▸ HERE. Pick from all 200 sessions in all 10 tracks, plus 22 Keynotes & General Sessions! Lunch is served two days. EXPIRES AUGUST 31, 2018. Ticket prices: ($1,295-Aug 31) ($1,495-Oct 31) ($1,995-Nov 12) ($2,500-Walk-in)
Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settlement products to hedge funds and investment banks. After, he co-founded a revenue cycle management company where he learned about Bitcoin and eventually Ethereal. Andrew's role at ConsenSys Enterprise is a mul...
Nicolas Fierro is CEO of MIMIR Blockchain Solutions. He is a programmer, technologist, and operations dev who has worked with Ethereum and blockchain since 2014. His knowledge in blockchain dates to when he performed dev ops services to the Ethereum Foundation as one the privileged few developers to work with the original core team in Switzerland.
René Bostic is the Technical VP of the IBM Cloud Unit in North America. Enjoying her career with IBM during the modern millennial technological era, she is an expert in cloud computing, DevOps and emerging cloud technologies such as Blockchain. Her strengths and core competencies include a proven record of accomplishments in consensus building at all levels to assess, plan, and implement enterprise and cloud computing solutions. René is a member of the Society of Women Engineers (SWE) and a m...
Digital Transformation and Disruption, Amazon Style - What You Can Learn. Chris Kocher is a co-founder of Grey Heron, a management and strategic marketing consulting firm. He has 25+ years in both strategic and hands-on operating experience helping executives and investors build revenues and shareholder value. He has consulted with over 130 companies on innovating with new business models, product strategies and monetization. Chris has held management positions at HP and Symantec in addition to ...
The challenges of aggregating data from consumer-oriented devices, such as wearable technologies and smart thermostats, are fairly well-understood. However, there are a new set of challenges for IoT devices that generate megabytes or gigabytes of data per second. Certainly, the infrastructure will have to change, as those volumes of data will likely overwhelm the available bandwidth for aggregating the data into a central repository. Ochandarena discusses a whole new way to think about your next...
CloudEXPO | DevOpsSUMMIT | DXWorldEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.
Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more busine...
All in Mobile is a place where we continually maximize their impact by fostering understanding, empathy, insights, creativity and joy. They believe that a truly useful and desirable mobile app doesn't need the brightest idea or the most advanced technology. A great product begins with understanding people. It's easy to think that customers will love your app, but can you justify it? They make sure your final app is something that users truly want and need. The only way to do this is by ...
DXWorldEXPO LLC announced today that Big Data Federation to Exhibit at the 22nd International CloudEXPO, colocated with DevOpsSUMMIT and DXWorldEXPO, November 12-13, 2018 in New York City. Big Data Federation, Inc. develops and applies artificial intelligence to predict financial and economic events that matter. The company uncovers patterns and precise drivers of performance and outcomes with the aid of machine-learning algorithms, big data, and fundamental analysis. Their products are deployed...