Industrial IoT Authors: William Schmarzo, Elizabeth White, Stackify Blog, Yeshim Deniz, SmartBear Blog

Related Topics: Industrial IoT, Microservices Expo

Industrial IoT: Article

Out of Step - NIEM and N-DEx

Two national data-sharing initiatives face major challenges

Since the horrific events of September 11, 2001, the federal government has intensified its efforts to improve communications, collaboration, and information sharing between government and private sector agencies at all levels. The task of creating a seamless system of data and communication between disparate agencies has faced both technological and political obstacles.

This article will look at two federal data-sharing initiatives: the National Information Exchange Model (NIEM) and the National Data Exchange (N-DEx). Each offers new information-sharing capabilities and each faces major challenges.

Is universal sharing of information between governmental agencies actually doable? NIEM is an ambitious new initiative that is taking a giant step towards making the dream of government-wide data interoperability a reality. NIEM is a joint effort of the Departments of Homeland Security and Justice. Its mission is to create the means to seamlessly exchange information electronically between multiple governmental agencies.

The task of bridging hundreds of formerly stove-piped systems is enormous. Every system has its own way of formatting data and defining the meaning of database terms. For instance, one system may use the term "FirstName" and another the term "FName" to specify a person's first name in a database. In other cases, different systems may use an identical label to represent different data. "CNum" may mean "case number" in one system and "catalog number" in another.

Clearly the development of common standards is central to the success of NIEM. To bridge the differences in business needs and data standards between domains, NIEM will build upon the commonalities of data elements to create Common Core and Universal Core Information Exchange Packages (IEPs). Domain-specific exchange standards will also be developed to address intradomain characteristics.

The National Information Exchange Model - Following Successful Footsteps
Shared Extensible Markup Language (XML) is a technology standard that allows information and services to be encoded with meaningful structure and semantics that facilitate information exchange. The Department of Justice's (DOJ) Infrastructure and Standards Working Group chose XML as the open standard for data exchanges in early 2002.

The Global Justice Information Sharing Initiative (Global) was then founded to create a Justice-specific implementation of XML. Global, a consortium of organizations that are members of the justice community, released The Global Justice XML Data Model (GJXDM) in February 2004.

One year later in February 2005, Steve Cooper, the CIO of the Department of Homeland Security (DHS), and Vance Hitch, the CIO of DOJ, jointly announced that both agencies had selected the GJXDM as the standard for data sharing and interoperability by their agencies and that a new federal initiative, the National Information Model Exchange Model, would be created.

NIEM will be designed to bridge the divides between Justice, Health, Transportation, Intelligence, and other domains. It will build upon the successful implementation of the Global Justice XML Data Model (GJXML), which has already provided the means to tie together data between all federal, state, local, and tribal agencies in the justice community.

One major objective the NIEM is to prevent the creation of XML information silos. While GJXDM has become the standard for interoperability in the justice domain, other domains such as homeland security, emergency management, health and human services, and transportation each have unique business rules and information-sharing standards that are domain-specific. To prevent domain silos, NIEM will provide the framework, architecture, security, and metadata controls necessary to assure that interoperability between domains will not be compromised. It will also develop domain-data dictionaries and schemas to meet the business needs of each community.

NIEM's success is dependent upon continued political support and funding. It also requires the acceptance of a new sharing paradigm among federal agencies and among federal, state, and local entities. Its foundation is based upon years of work by a large number of organizations that have established credentials and that support the program. It is built upon the successful implementation of the GJXDM and its future offers a major step forward in cross-domain electronic data sharing. For all of these reasons the National Information Exchange Model has a high probability of success. The same cannot be said for the National Data Exchange (see Figure 1).

The National Data Exchange - Trying to Find Its Footing
The Department of Justice created the Law Enforcement Information Sharing Program (LEISP) in response to a presidential order from August of 2004 requiring federal law enforcement and domestic security agencies to cooperate more closely with state and local police. In response, the Federal Bureau of Investigation (FBI) developed its National Information Sharing Strategy (NISS). The National Data Exchange (N-DEx) and Regional Data Exchanges (R-DEx) are two NISS initiatives by the FBI to improve the sharing of information among law enforcement agencies. Information concerning criminal suspects including, their method of operation, the vehicles they use, their accomplices, phone numbers, addresses, weapons used, and many additional factors are shared locally and in some cases regionally, but currently there is no national system for automatically sharing investigative data from law enforcement records-management software.

N-DEx Will Be a Criminal Investigative Sharing System
N-DEx has been in pilot testing since June 2004 with the Uniform Crime Reporting (UCR) State Repository, West Virginia State Police, the Cabell County, West Virginia, Sheriff's Office, and police departments in Marietta, Georgia, and Alexandria, Virginia. Based on the lessons learned from the pilot sites and input from focus groups and national law enforcement organizations, the Bureau is currently completing its Concept of Operations Plan, which will guide the implementation process.

N-DEx will correlate data from all major FBI databases and provide the ability to execute nationwide inquiries from a single access point. It offers for the first time a national system that can be used to identify crime trends, link cases, and solve crimes.

In 2005 the FBI implemented two Regional Data Exchanges (R-DEx), the first in St. Louis and the second in Seattle. R-DEx will give state, local, and tribal law enforcement appropriate access to federal investigative and intelligence information on a regional basis. The FBI, US Marshals, and the Bureau of Prisons are currently sharing investigative reports with the local participants. The DEA and ATF are also expected to add information from their databases to the system shortly. Twelve to 18 additional R-DEx projects are currently being planned for implementation over the next several years.

To many in the law enforcement community, the sharing of regional criminal data is of more value than a national system. This is due to the fact that the overwhelming majority of criminal investigations are local or regional in scope. Programs such JNET, ARGUS, and FINDERS have demonstrated the value of such regional approaches.

N-DEx Will Be an Incident-Based Collection Instrument Used for Crime Reporting
In addition to serving as a criminal investigation data-sharing tool, N-DEx has two additional major objectives: the electronic reporting of crimes and reporting of crime statistics to the FBI.

State and local agencies have been reporting monthly counts of key offenses and arrests to the FBI through the Uniform Crime Reporting (UCR) Program since 1929. This program was greatly expanded in 1992 through the National Incident-Based Reporting Systems (NIBRS), which collects more details on more categories of crime such as hate crime, sexual assault among young victims, and kidnapping of juveniles.

Both programs are still used for the collection of crime statistics. The information that is reported to the FBI under these programs is collected from data that is contained within police reports. Most computerized law enforcement records-management systems produce UCR or NIBRS reports electronically. The reports are then sent via paper or electronically to the FBI for publication.

As a national program, NIBRS is widely regarded as a failed effort. One measure of its failure is that by 2004, 14 years after its introduction, only 23 states were fully or partially participating in submitting data. The proposal to include crime reporting as part of the N-DEx program has meet with confusion and concern by local law enforcement agencies. They want to know if N-DEx will replace UCR and NIBRS, and who is going to pay for the cost for changing their software to provide the new functionality.

The N-DEx program has used a consensus-building process called the National Consensus Process (NCP) to help define the scope of the program. Law enforcement and other criminal justice entities representing both large and small agencies have participated in the consensus process as members of focus groups. Several focus group meetings have been held to gather input and build consensus with regard to the definition of the N-DEx program. However, while individual focus groups are important in defining the program's objectives, the input and approval of the national law enforcement associations must be sought before the political support necessary to carry this project forward is obtained.

More Stories By Neil Kurlander

Neil Kurlander is the vice president of Public Sector Solutions for Asynchrony Solutions (www.asolutions.com). His dual careers in public safety and technology span four decades. His advocacy for the use of technology by law enforcement resulted in his appointment as chair of technology committees on the national, state, and local levels. Neil is a life member of the International Association of Chiefs of Police and currently serves on the association?s Communication and Technology Committee. He also chairs the Law Enforcement Information Technology Standards Advisory Committee of the Integrated Justice Information Systems Institute.

Comments (2)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

IoT & Smart Cities Stories
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.
Digital Transformation is much more than a buzzword. The radical shift to digital mechanisms for almost every process is evident across all industries and verticals. This is often especially true in financial services, where the legacy environment is many times unable to keep up with the rapidly shifting demands of the consumer. The constant pressure to provide complete, omnichannel delivery of customer-facing solutions to meet both regulatory and customer demands is putting enormous pressure on...
IoT is rapidly becoming mainstream as more and more investments are made into the platforms and technology. As this movement continues to expand and gain momentum it creates a massive wall of noise that can be difficult to sift through. Unfortunately, this inevitably makes IoT less approachable for people to get started with and can hamper efforts to integrate this key technology into your own portfolio. There are so many connected products already in place today with many hundreds more on the h...
The standardization of container runtimes and images has sparked the creation of an almost overwhelming number of new open source projects that build on and otherwise work with these specifications. Of course, there's Kubernetes, which orchestrates and manages collections of containers. It was one of the first and best-known examples of projects that make containers truly useful for production use. However, more recently, the container ecosystem has truly exploded. A service mesh like Istio addr...
Digital Transformation: Preparing Cloud & IoT Security for the Age of Artificial Intelligence. As automation and artificial intelligence (AI) power solution development and delivery, many businesses need to build backend cloud capabilities. Well-poised organizations, marketing smart devices with AI and BlockChain capabilities prepare to refine compliance and regulatory capabilities in 2018. Volumes of health, financial, technical and privacy data, along with tightening compliance requirements by...
Charles Araujo is an industry analyst, internationally recognized authority on the Digital Enterprise and author of The Quantum Age of IT: Why Everything You Know About IT is About to Change. As Principal Analyst with Intellyx, he writes, speaks and advises organizations on how to navigate through this time of disruption. He is also the founder of The Institute for Digital Transformation and a sought after keynote speaker. He has been a regular contributor to both InformationWeek and CIO Insight...
Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settlement products to hedge funds and investment banks. After, he co-founded a revenue cycle management company where he learned about Bitcoin and eventually Ethereal. Andrew's role at ConsenSys Enterprise is a mul...
To Really Work for Enterprises, MultiCloud Adoption Requires Far Better and Inclusive Cloud Monitoring and Cost Management … But How? Overwhelmingly, even as enterprises have adopted cloud computing and are expanding to multi-cloud computing, IT leaders remain concerned about how to monitor, manage and control costs across hybrid and multi-cloud deployments. It’s clear that traditional IT monitoring and management approaches, designed after all for on-premises data centers, are falling short in ...
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more busine...