FAQs
The final operational release of NIEM Version 3.1 has been published and is now available. NIEM 3.1 is a minor release that incorporates user needs since NIEM 3.0 was launched in November 2013.
NIEM 3.1 highlights:
One new domain has added content to the model for the first time:
Welcome, Human Services Domain!
Six existing domains have updated their model content:
Biometrics
Children, Youth, and Family Services (CYFS)
International Trade
Justice
Maritime?
Military Operations (MilOps)
Two core supplements that apply to NIEM Core 3.0 are included:
Supplement 3.0.1 - Incorporates all U.S. State codes into U.S. Postal Service (USPS) code list-including Alabama (AL) that was previously omitted.
Supplement 3.0.2 - Incorporates the following updates:
Adds correct constraint for Longitude MGRSCoordinateType (Military Grid Reference System); increases precision.
Adds PersonUnionStatus (for Person associations).
Adds new values added by authoritative sources to the following code lists:
dot_hazmat
dod_jcs-pub2.0
dea_ctlsub
census_uscounty
NIEM 3.1 marks the first time we have exercised the new Version 3.0 architecture that allows:
(1) easy adjustments to code lists and
(2) supplements to core without the need to wait for a major release cycle.
The NIEM 3.1 Final Version Package is available at: https://niem.github.io/niem-releases/
For most information exchange applications, it is preferable not to use the entire schema, but rather a "subset" schema containing only those types and elements needed for that exchange.
When creating a subset, it is important to follow a small set of rules, to ensure that the subset is the functional equivalent (for a particular application) of the full GJXDM or NIEM. The most important subset rule is that an instance that validates against a correct GJXDM or NIEM subset will also validate against the entire GJXDM or NIEM schema. This and other rules for subset schemas have been published online (see link below), so it is certainly possible for schema designers to create subset schemas "by hand" by following these rules.
However, after the idea of subset schemas was introduced, manual creation of subset schemas proved cumbersome, tedious, and error-prone. This was mostly because when selecting a type or element for the subset, there are often other types and elements on which the selected type or element is dependent upon; these dependencies have to be included in the subset for it to be valid. While sometimes difficult to remember, the rules for dependencies are nonetheless consistent and deterministic. Luckily, as a result, the rules lend themselves to implementation in a tool that checks the validity of the subset schema.
GJXDM Validator: https://bja.ojp.gov/program/it/national-initiatives/niem (Note: Currently there is no Validator for NIEM, but it can be validated using third party tools)
Rules for Schema Subsets: https://www.niem.gov/documentsdb/Documents/Technical/NIEM-NDR-1-3.pdf
Key Words are navigational terms to help identify the NIEM name for required components. There are often many ways to name a particular data element. NIEM prohibit the use of synonyms within the XML schema specification. However, the database from which the schema is generated may contain additional navigational terms to help identify the NIEM names for required components. We refer to such navigational terms as key words. Like context data elements, key words do not appear in the schema but do appear in the dictionary database.
Yet, key words differ from context data elements in that they are not represented by inherited properties (which usually have generic names). Instead, they are simply terms or common abbreviations that are near in meaning to, synonymous with, or suggestive of NIEM property names. Key words often come from other data models, dictionaries, or schemas. The following is an example of "Key Words" and the components they might link to:
KEY WORD
LINKS TO
Automobile
Vehicle
Car
Vehicle
ChildFlag
JuvenileIndicator
Fine
DisciplinaryActionFee
Key words capability is available in the NIEM SSGT.
NIEM SSGT: http://niem.gtri.gatech.edu/niemtools/ssgt/index.iepd
The namespace is used for schema version control, in that the contents of GJXDM or NIEM (elements, types, attributes, and attribute groups, as well as the definitions of all of these things) will never change for a given namespace identifier. That is, for example, the contents of the GJXDM namespace at version 3.0.3 will always be the same for that namespace. And any changes to the data model will require a new namespace identifier.
Example of a namespace: https://bja.ojp.gov/program/it/national-initiatives/gjxdm
Directory of all GJXDM namespace releases: https://bja.ojp.gov/program/it/national-initiatives/gjxdm
NIEM namespaces: http://www.niem.gov/niem/namespaces
It is possible to extend GJXDM and NIEM, like any other XML markup language. There are several ways that a local schema might do this. A simple set of examples has been prepared that illustrates various extension methods.
One of these methods is based on the World Wide Web consortium (W3C) rules for extension of XML Schema types. W3C Schema rules for type extension allow many possibilities. However, type extension within the GJXDM and NIEM is intended to maintain a class hierarchy of objects by adhering to a more restrictive set of subclass rules.
To ensure the integrity, consistency, and meaning of the class (inheritance) hierarchy, the following rules for type extension must be followed:
1) A derived type may add (by extension) additional fields (elements/attributes) to its base type.
2) A derived type may restrict one or more fields of its base type, but only so that a derived field is a subset of the field of the base type. For example, a derived type may:
Restrict an enumeration from a large set of options to a smaller set of options, as long as every option in the derived set appears in the base set
Remove a field of the base type only if the field is optional in the base type
Require a field to appear only if the field is optional or required to appear in the base type
3) A derived type may not modify a field of its base type such that it violates the constraints of its base type. For example, a derived type may not:
Add additional enumerations to a field
Remove a field that is required by its base type
Modify the type of a field of its base type
W3C rules for Extension XML Schema Types:http://www.w3.org/tr/xmlschema-0/#derivext
There are several tools available for validation of WSDL that imports a GJXDM or NIEM subset. WSDL stands for Web Services Description Language.
WSDL is a document written in XML. The document describes a Web service. It specifies the location of the service and the operations (or methods) the service exposes. Both xsdvalid and the built-in schema validator in XMLBeans (where the validator is a part of the schema object model) can be used in validating subsets generated by the SSGT (and the entire GJXDM or NIEM distribution itself). Similar tools work for this purpose in the .NET environment as well.
If one wants to validate the schema that defines the types used by elements in the WSDL document, then a schema validation tool like xsdvalid should work fine. However, if one is looking to validate the WSDL document as well, but can't find a tool that will do it, then one workaround would be to generate client stubs using something like Apache Axis' wsdl2java utility and look for errors in that process. The process is not elegant, and one would wind up with the client stub code. Yet Axis wsdl2java has very high fidelity to the WSDL spec, and it would definitely do the job.
It is not a good idea to workaround the problem by pasting all of the GJXDM or NIEM subset schema elements into a single schema, since that would necessarily mean collapsing everything into a single namespace. Altering the namespace in which schema elements appear is a violation of the rules for GJXDM and NIEM subsets.
XMLBeans:http://xmlbeans.apache.org/
One key concept is that GJXDM and NIEM were designed for information exchange, and have no direct bearing on how one would implement a database that is part of a sending or receiving system; the database should be designed to hold the data required for the system's business purpose. At this time there are no web services that have been identified or documented for use with the GJXDM and NIEM surrounding the DB implementation, although work is being done in that area.
However, a number of organizations have had success deploying web services to exchange information using the GJXDM. Some have used web services conformant with the WS-I Basic Profile. XML messaging standards (depending on what you mean by ''messaging standards'') for the GJXDM are emerging but are not complete, although again, work is being done in that area. Organizations are developing Information Exchange Package Documentation to specify information exchange packages such and Incident Report, Court Filing, etc.
Organizations such as NLETS have released GJXDM-conformant message specifications for the information exchanges they support. An example with SQL is the data exchange project WENET, where a couple of the service points use the MS SQL server XML services to directly extract GJXDM data from SQL. Currently it is only in the early stages of development.
The hardest part seems to be creating a GJXDM or NIEM complaint sub-schema that SQL will like. The sub-sets strip out all of the imports and make them local, redesign some of the global element model and generally look different from the GJXDM or NIEM but still validate. Then they are marked up with SQL-XML tags/attributes specifying tables/fields and relationships. This is demonstrated with simple elements and is working towards it with Person, Warrant, and some others.
The most popular RDBMS and IDE tools, including those from Microsoft and Oracle, can support DB implementations based on GJXDM or NIEM. These tools provide capabilities for parsing XML and moving XML data in and out of databases. However, depending on ones particular database, it may be necessary to use XSLT to transform between a GJXDM- or NIEM-conformant XML schema and an XML schema that works with your database.
WENET Data Exchange Project: http://www.whatcomcounty.us/515/Whatcom-Exchange-Network-WENET-Project
There are a large number of transformations which could be performed on the GJXDM and NIEM to create a subset, and many of those possible transformations are not recommended. It is important that the generated schema subset adhere to the rules for schema subsets.
The following transformations should not be performed:
Do not omit any components that are required.
Do not rearrange or change the order of elements appearing in schemas.
Do not change the base type of any type
Do not change the type of any element or attribute
Do not make global components local
Do not change the namespace of elements, attributes, types, etc.
Do not rename components
Do not flatten structures
Do not change any namespace URI
Do not change the relative file path location of files (i.e. .../3.0/jxdm.xsd)
Do not do anything not specified by this document
There is no perfect strategy for handling incomplete or partial dates within the GJXDM and NIEM. However, different options do exist for different circumstances. Most of the date fields are of type ''xsd:date'' whereas for example a number of our entities can include only the year. As an example, consider the case where a license plate registration year is stored only as a year. In GJXDM and NIEM there is VehicleRegistration/RegistrationEffectiveDate. For this example there are two options:
1. Just use 01-01-XXXX and let the receiver know out of band that the month and day are unreliable. An advantage of this approach is that it doesn't require any schema changes and thus the data are more easily exchanged. Disadvantage is that it's rather inelegant and confusing.
2. Extend the schema to add a field of type ''gYear'' in the appropriate spot. This would require creating a local extension of PropertyRegistration, then a local extension of VehicleRegistration to use the PropertyRegistration base type. (VehicleRegistration could be extended directly but just for the sake of argument I'll put it in exactly the same hierarchy.) Then if VehicleRegistration is encapsulated in a Vehicle, type substitution must be used in the appropriate Vehicle entity. The advantage is that this way is more exact; disadvantage is that it requires much more work and has local extensions which harm interoperability.
NLETS dealt with a similar problem in which the dates only included the year, but because of the legacy data there was also the possibility of receiving several other textual values, as well as an inability to know the data while creating XML. Therefore they created an element in our namespace of type TextType and implemented using cascading extension. If properly documented as a business rule, the full date element can probably be used in the Registration situation.
However, one should be wary of implications if more exact data could be provided in the future. One runs the risk of having some exact dates, and some with 01-01 as a space holder. Then it is impossible to know if the registration was actually as of January 1, or if the Month/Day were unknown. Keep in mind that type substitution may not be appropriate here because one can only substitute with a type derived from the expected type.
Another option which would allow avoiding extension but more clearly communicating the data in the element is to leverage the comment Text metadata attribute to communicate that the month/day is unknown. The choice depends largely on the implementation circumstances. It's generally better to ensure that the content of the element will be unambiguous and err on the side of an additional extension rather than risk misinterpretation.
A GJXDM or NIEM component is any type or property defined in the models. Types translate to XML Schema types and properties translate to XML schema elements or attributes from core, domains, code list schemas, and external standard adapters. There are currently over 4000 components in NIEM 2.0 and over 2000 components in GJXDM 3.03. There will be more components to be add in the new version NIEM 2.1 that pulished in October 2009.
GJXDM and NIEM follow the Federal XML Developer's Guidelines, ebXML, ISO/IEC Standard 11179, and best practices. Abbreviations and acronyms are highly discouraged within data element names, except when commonly understood, rarely confused, widely accepted, agreed upon, and documented. Authorized abbreviations are listed and defined in the glossary of the documentation.
Federal XML Developer's Guide: http://xml.fido.gov/documents/in_progress/developersguide.pdf
GJXDM and NIEM do not contain elements for enforcing security requirements for information exchanges. Enforcing these requirements necessitates leveraging other standards, which will differ depending on the overall architecture of the exchange. For example, in web services, many exchange security requirements are addressed by the Web Services Security (WS-Security) standard from OASIS, and the Web Services Interoperability Organization (WS-I) Basic Security Profile.
The OASIS LegalXML Court Filing technical committee has addressed many of these information security requirements in its standards development work. This work is recommended as an example of how to leverage open security standards in GJXDM- and NIEM-conformant exchanges.
In accordance with ISO 11179, each GJXDM and NIEM tag name consists of several parts:
• Object class term: The object class term represents the specific real-world object to which the property is applicable.
• Property term: The property term is a plain-language summary of the quality that the property represents. Example property terms would include Hair Color, Hair Appearance, and Tax Identifier.
• Representation term: The representation term describes, from a very high level, the form of the data represented. This term is taken from a list of “ebXML” representation terms. Representation terms not on the list are not valid. The point of the representation term is not to specify the exact type of the represented value; It is instead intended to give the person reading the property name a clue as to what the property is for and what it might mean.
The Global JXDM is unique within government as an XML vocabulary that has truly been created from the ground up. Practitioners from a variety of organizations came together to create a data dictionary that would allow the entire justice enterprise to share information with a common structure. This enables exchanges to be built that serve many purposes and eliminates the point-to-point inefficiencies of the past. The success of Global JXDM has standardized many disparate systems across the criminal justice domain and the concept is now being extended on a national level. Global JXDM will continue to operate under the guidance of DOJ Bureau of Justice Affairs (BJA), and NIEM will serve as an umbrella organization with a much larger scope. DOJ Office of the CIO is responsible for NIEM, in partnership with the DHS Office of the CIO.
In NIEM, the Global JXDM is represented in the justice domain. NIEM not only includes the justice domain, but also represents others, such as intelligence and emergency and disaster management. NIEM provides the organizations involved in these domains with the data model needed to create their information exchanges, to create and share information, and to get a head start in implementing its own exchanges. NIEM actively encourages Federal agencies while equally focusing on state and local contributors. This is possible through an emphasis on component-based resources that are reusable and portable to any organization or platform. NIEM will serve as an umbrella for existing domains, such as justice and homeland security, to work side by side in developing information exchange capabilities and ensure that technology will never again be a barrier to the public's safety and well being.
Conceptually, the IEPD lifecycle in NIEM (as defined in the NIEM Concept of Operations) is very similar to the IEPD lifecycle followed by many IEPD developers in GJXDM. The GJXDM lifecycle is documented in the GJXDM Users Guide.
Both lifecycles call for the basic steps of understanding the contextual business scenarios or context of exchange, documenting exchange requirements, modeling the information being exchanged, mapping the information model to GJXDM or NIEM, and building schemas from the mapping. Both lifecycles suggest (the NIEM lifecycle does so explicitly) that IEPDs should be published and registered so that others may find and reuse them.
The NIEM lifecycle references supporting tools that were not available in the GJXDM lifecycle, such as the NIEM Naming and Design Rules (NDR) and the NIEM Component Mapping Template (CMT).
The GJXDM lifecycle makes more explicit reference to the benefit of logical domain modeling as a way of capturing information requirements that are independent of the XML vocabulary. The domain model represents the requirements that are mapped to GJXDM (or NIEM).