FAQs
NIEM 3.1 Beta Version is a final test version of NIEM 3.1. This minor upgrade version will associate with our NIEM NDR 3.0 and MPD 3.0.
This NIEM 3.1 Beta Version will available for public review and comment from March 23 to April 6 of 2015. This final version of NIEM 3.1 will expect to be released in May/June 2015. This version will incorporate errors or bugs that users identified in the NIEM 3.0 version last year.
Here are some of the updates that NIEM 3.1 Beta Version includes:
Updated model content in six domains:
1. Justice
2. Millitary Operations (MilOps)
3. Biometrics
4. Maritime
5. Children, Youth, and Family Services (CYFS)
6. International Trade
The addition of the Human Services domain—they are adding content in the model for the first time!
Two core supplements:
1. To correct a code list associated with NIEM 3.0
2. To increase the precision of MGRSCoordinateType (Military Grid Reference System)
The NIEM 3.1 Beta Version Package is available at: https://www.niem.gov/about-niem/news/niem-version-31-now-available
The page for NIEM 3.1 Beta Version comment submissions is: https://www.niem.gov/about-niem/news/niem-31-beta1-available-review
The Geospatial Enhancement for NIEM (Geo4NIEM) initiative is a collaboration of the NIEM Program Management Office (PMO), the Open Geospatial Consortium (OGC), the Department of Homeland Security, and the Program Manager for the Information Sharing Environment (PM-ISE) to enhance NIEM's geospatial exchange capabilities.
The objectives of the Geo4NIEM initiative are to:
Develop recommendations for the inclusion and standard use of embedded geography markup language (GML) with NIEM Information Exchange Package Documentations (IEPDs).
Develop recommendations for the standardized use of Naming and Design Rules and the use of adaptors (e.g., NIEM wrapper for GML).
Test and demonstrate use of a standardized embedded GML and adaptors within NIEM IEPDs.
Develop architecture documentation and a fact sheet for the use of embedded GML and adaptors for use with NIEM IEPDs.
Develop recommendations for the inclusion of a Geospatial Domain within NIEM.
Geospatial information technologies are increasingly a foundation for supporting homeland security, law enforcement, emergency management, and public safety missions in the U.S. While these technologies rely upon much of the same data, they are typically developed in silos within a specific mission area. As a result, data duplication and data exchange delays occur. However, mission partners could benefit from shared access to the common operating data and services used within these geospatial systems if they were exposed and exchanged in open standards. Hence the need for enhancing NIEM’s geospatial exchange capabilities in order to significantly improve inter-government information sharing of this critical data source.
The Geo4NIEM initiative culminated in a demonstration of how enhanced geospatial capabilities enable situational awareness through the ability to identify, process, and comprehend critical information during an incident. More information and videos of the demonstration can be found here.
The Geo4NIEM initiative provided eight recommendations (specific to GML adapters in NIEM) for consideration in the NIEM 3.0 update; seven recommendations were implemented and one recommendation had no impact on the model itself. Overall, no significant changes were recommended to the NIEM technical architecture. The GML adapters are defined in the geospatial.xsd schema, a NIEM reference schema. In general, the standard adapter mechanism should be employed to import valid GML content into NIEM-conformant data structures.
An engineering report summarizing the findings and recommendations of the Geo4NIEM initiative was published in November of 2013. You can read more about the findings and download the engineering report here.
Documents
https://bja.ojp.gov/media/document/29986
The final operational release of NIEM Version 3.1 has been published and is now available. NIEM 3.1 is a minor release that incorporates user needs since NIEM 3.0 was launched in November 2013.
NIEM 3.1 highlights:
One new domain has added content to the model for the first time:
Welcome, Human Services Domain!
Six existing domains have updated their model content:
Biometrics
Children, Youth, and Family Services (CYFS)
International Trade
Justice
Maritime?
Military Operations (MilOps)
Two core supplements that apply to NIEM Core 3.0 are included:
Supplement 3.0.1 - Incorporates all U.S. State codes into U.S. Postal Service (USPS) code list-including Alabama (AL) that was previously omitted.
Supplement 3.0.2 - Incorporates the following updates:
Adds correct constraint for Longitude MGRSCoordinateType (Military Grid Reference System); increases precision.
Adds PersonUnionStatus (for Person associations).
Adds new values added by authoritative sources to the following code lists:
dot_hazmat
dod_jcs-pub2.0
dea_ctlsub
census_uscounty
NIEM 3.1 marks the first time we have exercised the new Version 3.0 architecture that allows:
(1) easy adjustments to code lists and
(2) supplements to core without the need to wait for a major release cycle.
The NIEM 3.1 Final Version Package is available at: https://niem.github.io/niem-releases/
For most information exchange applications, it is preferable not to use the entire schema, but rather a "subset" schema containing only those types and elements needed for that exchange.
When creating a subset, it is important to follow a small set of rules, to ensure that the subset is the functional equivalent (for a particular application) of the full GJXDM or NIEM. The most important subset rule is that an instance that validates against a correct GJXDM or NIEM subset will also validate against the entire GJXDM or NIEM schema. This and other rules for subset schemas have been published online (see link below), so it is certainly possible for schema designers to create subset schemas "by hand" by following these rules.
However, after the idea of subset schemas was introduced, manual creation of subset schemas proved cumbersome, tedious, and error-prone. This was mostly because when selecting a type or element for the subset, there are often other types and elements on which the selected type or element is dependent upon; these dependencies have to be included in the subset for it to be valid. While sometimes difficult to remember, the rules for dependencies are nonetheless consistent and deterministic. Luckily, as a result, the rules lend themselves to implementation in a tool that checks the validity of the subset schema.
GJXDM Validator: https://bja.ojp.gov/program/it/national-initiatives/niem (Note: Currently there is no Validator for NIEM, but it can be validated using third party tools)
Rules for Schema Subsets: https://www.niem.gov/documentsdb/Documents/Technical/NIEM-NDR-1-3.pdf
Key Words are navigational terms to help identify the NIEM name for required components. There are often many ways to name a particular data element. NIEM prohibit the use of synonyms within the XML schema specification. However, the database from which the schema is generated may contain additional navigational terms to help identify the NIEM names for required components. We refer to such navigational terms as key words. Like context data elements, key words do not appear in the schema but do appear in the dictionary database.
Yet, key words differ from context data elements in that they are not represented by inherited properties (which usually have generic names). Instead, they are simply terms or common abbreviations that are near in meaning to, synonymous with, or suggestive of NIEM property names. Key words often come from other data models, dictionaries, or schemas. The following is an example of "Key Words" and the components they might link to:
KEY WORD
LINKS TO
Automobile
Vehicle
Car
Vehicle
ChildFlag
JuvenileIndicator
Fine
DisciplinaryActionFee
Key words capability is available in the NIEM SSGT.
NIEM SSGT: http://niem.gtri.gatech.edu/niemtools/ssgt/index.iepd
The namespace is used for schema version control, in that the contents of GJXDM or NIEM (elements, types, attributes, and attribute groups, as well as the definitions of all of these things) will never change for a given namespace identifier. That is, for example, the contents of the GJXDM namespace at version 3.0.3 will always be the same for that namespace. And any changes to the data model will require a new namespace identifier.
Example of a namespace: https://bja.ojp.gov/program/it/national-initiatives/gjxdm
Directory of all GJXDM namespace releases: https://bja.ojp.gov/program/it/national-initiatives/gjxdm
NIEM namespaces: http://www.niem.gov/niem/namespaces
It is possible to extend GJXDM and NIEM, like any other XML markup language. There are several ways that a local schema might do this. A simple set of examples has been prepared that illustrates various extension methods.
One of these methods is based on the World Wide Web consortium (W3C) rules for extension of XML Schema types. W3C Schema rules for type extension allow many possibilities. However, type extension within the GJXDM and NIEM is intended to maintain a class hierarchy of objects by adhering to a more restrictive set of subclass rules.
To ensure the integrity, consistency, and meaning of the class (inheritance) hierarchy, the following rules for type extension must be followed:
1) A derived type may add (by extension) additional fields (elements/attributes) to its base type.
2) A derived type may restrict one or more fields of its base type, but only so that a derived field is a subset of the field of the base type. For example, a derived type may:
Restrict an enumeration from a large set of options to a smaller set of options, as long as every option in the derived set appears in the base set
Remove a field of the base type only if the field is optional in the base type
Require a field to appear only if the field is optional or required to appear in the base type
3) A derived type may not modify a field of its base type such that it violates the constraints of its base type. For example, a derived type may not:
Add additional enumerations to a field
Remove a field that is required by its base type
Modify the type of a field of its base type
W3C rules for Extension XML Schema Types:http://www.w3.org/tr/xmlschema-0/#derivext
There are several tools available for validation of WSDL that imports a GJXDM or NIEM subset. WSDL stands for Web Services Description Language.
WSDL is a document written in XML. The document describes a Web service. It specifies the location of the service and the operations (or methods) the service exposes. Both xsdvalid and the built-in schema validator in XMLBeans (where the validator is a part of the schema object model) can be used in validating subsets generated by the SSGT (and the entire GJXDM or NIEM distribution itself). Similar tools work for this purpose in the .NET environment as well.
If one wants to validate the schema that defines the types used by elements in the WSDL document, then a schema validation tool like xsdvalid should work fine. However, if one is looking to validate the WSDL document as well, but can't find a tool that will do it, then one workaround would be to generate client stubs using something like Apache Axis' wsdl2java utility and look for errors in that process. The process is not elegant, and one would wind up with the client stub code. Yet Axis wsdl2java has very high fidelity to the WSDL spec, and it would definitely do the job.
It is not a good idea to workaround the problem by pasting all of the GJXDM or NIEM subset schema elements into a single schema, since that would necessarily mean collapsing everything into a single namespace. Altering the namespace in which schema elements appear is a violation of the rules for GJXDM and NIEM subsets.
XMLBeans:http://xmlbeans.apache.org/
One key concept is that GJXDM and NIEM were designed for information exchange, and have no direct bearing on how one would implement a database that is part of a sending or receiving system; the database should be designed to hold the data required for the system's business purpose. At this time there are no web services that have been identified or documented for use with the GJXDM and NIEM surrounding the DB implementation, although work is being done in that area.
However, a number of organizations have had success deploying web services to exchange information using the GJXDM. Some have used web services conformant with the WS-I Basic Profile. XML messaging standards (depending on what you mean by ''messaging standards'') for the GJXDM are emerging but are not complete, although again, work is being done in that area. Organizations are developing Information Exchange Package Documentation to specify information exchange packages such and Incident Report, Court Filing, etc.
Organizations such as NLETS have released GJXDM-conformant message specifications for the information exchanges they support. An example with SQL is the data exchange project WENET, where a couple of the service points use the MS SQL server XML services to directly extract GJXDM data from SQL. Currently it is only in the early stages of development.
The hardest part seems to be creating a GJXDM or NIEM complaint sub-schema that SQL will like. The sub-sets strip out all of the imports and make them local, redesign some of the global element model and generally look different from the GJXDM or NIEM but still validate. Then they are marked up with SQL-XML tags/attributes specifying tables/fields and relationships. This is demonstrated with simple elements and is working towards it with Person, Warrant, and some others.
The most popular RDBMS and IDE tools, including those from Microsoft and Oracle, can support DB implementations based on GJXDM or NIEM. These tools provide capabilities for parsing XML and moving XML data in and out of databases. However, depending on ones particular database, it may be necessary to use XSLT to transform between a GJXDM- or NIEM-conformant XML schema and an XML schema that works with your database.
WENET Data Exchange Project: http://www.whatcomcounty.us/515/Whatcom-Exchange-Network-WENET-Project
There are a large number of transformations which could be performed on the GJXDM and NIEM to create a subset, and many of those possible transformations are not recommended. It is important that the generated schema subset adhere to the rules for schema subsets.
The following transformations should not be performed:
Do not omit any components that are required.
Do not rearrange or change the order of elements appearing in schemas.
Do not change the base type of any type
Do not change the type of any element or attribute
Do not make global components local
Do not change the namespace of elements, attributes, types, etc.
Do not rename components
Do not flatten structures
Do not change any namespace URI
Do not change the relative file path location of files (i.e. .../3.0/jxdm.xsd)
Do not do anything not specified by this document
There is no perfect strategy for handling incomplete or partial dates within the GJXDM and NIEM. However, different options do exist for different circumstances. Most of the date fields are of type ''xsd:date'' whereas for example a number of our entities can include only the year. As an example, consider the case where a license plate registration year is stored only as a year. In GJXDM and NIEM there is VehicleRegistration/RegistrationEffectiveDate. For this example there are two options:
1. Just use 01-01-XXXX and let the receiver know out of band that the month and day are unreliable. An advantage of this approach is that it doesn't require any schema changes and thus the data are more easily exchanged. Disadvantage is that it's rather inelegant and confusing.
2. Extend the schema to add a field of type ''gYear'' in the appropriate spot. This would require creating a local extension of PropertyRegistration, then a local extension of VehicleRegistration to use the PropertyRegistration base type. (VehicleRegistration could be extended directly but just for the sake of argument I'll put it in exactly the same hierarchy.) Then if VehicleRegistration is encapsulated in a Vehicle, type substitution must be used in the appropriate Vehicle entity. The advantage is that this way is more exact; disadvantage is that it requires much more work and has local extensions which harm interoperability.
NLETS dealt with a similar problem in which the dates only included the year, but because of the legacy data there was also the possibility of receiving several other textual values, as well as an inability to know the data while creating XML. Therefore they created an element in our namespace of type TextType and implemented using cascading extension. If properly documented as a business rule, the full date element can probably be used in the Registration situation.
However, one should be wary of implications if more exact data could be provided in the future. One runs the risk of having some exact dates, and some with 01-01 as a space holder. Then it is impossible to know if the registration was actually as of January 1, or if the Month/Day were unknown. Keep in mind that type substitution may not be appropriate here because one can only substitute with a type derived from the expected type.
Another option which would allow avoiding extension but more clearly communicating the data in the element is to leverage the comment Text metadata attribute to communicate that the month/day is unknown. The choice depends largely on the implementation circumstances. It's generally better to ensure that the content of the element will be unambiguous and err on the side of an additional extension rather than risk misinterpretation.
A GJXDM or NIEM component is any type or property defined in the models. Types translate to XML Schema types and properties translate to XML schema elements or attributes from core, domains, code list schemas, and external standard adapters. There are currently over 4000 components in NIEM 2.0 and over 2000 components in GJXDM 3.03. There will be more components to be add in the new version NIEM 2.1 that pulished in October 2009.
GJXDM and NIEM follow the Federal XML Developer's Guidelines, ebXML, ISO/IEC Standard 11179, and best practices. Abbreviations and acronyms are highly discouraged within data element names, except when commonly understood, rarely confused, widely accepted, agreed upon, and documented. Authorized abbreviations are listed and defined in the glossary of the documentation.
Federal XML Developer's Guide: http://xml.fido.gov/documents/in_progress/developersguide.pdf
GJXDM and NIEM do not contain elements for enforcing security requirements for information exchanges. Enforcing these requirements necessitates leveraging other standards, which will differ depending on the overall architecture of the exchange. For example, in web services, many exchange security requirements are addressed by the Web Services Security (WS-Security) standard from OASIS, and the Web Services Interoperability Organization (WS-I) Basic Security Profile.
The OASIS LegalXML Court Filing technical committee has addressed many of these information security requirements in its standards development work. This work is recommended as an example of how to leverage open security standards in GJXDM- and NIEM-conformant exchanges.