v5.4.0

Release Notes for Data Hub 5.4.0

Release Summary

Data Hub 5.4 introduces major features and improvements in these areas:

Hub Central

• Load
  Users can load CSV, XML, JSON, binary or text files with model-based faceting/sorting, including source type and source name facets
• Curation
  Users can map fields in source files to entity properties, including structured types (e.g. Address, which consists of Street, City, State)
  Users can run any type of step, and view custom step settings
• Curation: Smart Mastering - Users can now configure match and merge steps in Hub Central using a simple, graphical UI:
  Add thresholds and define match rulesets, and adjust with a slider
  Define merge strategies and add merge rules
  View possible combinations of Matched Rulesets
  Change match configuration settings and merge step settings
• Modeling - Users can create, edit, delete and save entity types and properties
  Create simple (int, string, etc.) or deeply-nested structured properties, including multi-value arrays
  Create relationship property types, and link them at the entity type level
  Denote when properties have multiple values of their type
  Enable sorting and faceting for properties
• Data Exploration - Users can now search and explore with raw data in addition to curated data.
  Explore raw and curated data with model-based faceting/sorting
  Search and explore raw and curated data via table and snippet views
  Facet and filter data on properties defined in the model
  Save search queries
  Export searches in CSV format directly via the UI

Other Improvements

• Smart Mastering with Gradle
• Single value structured property matching (select 1 property)
• Single-value structured property matching (select multiple properties)
• Multi-value structured property matching
• Multi-value with multiple structured type matching

Step Flexibility

• Custom steps can now process more than URIs. You can now set sourceQuery to a script that retrieves items of different data types to pass to your custom step module.
• New step interceptors allow you to manipulate processed documents before they are persisted. They can be configured for any step and run in the same step transaction. One application of this is adding permissions based on document contents.

Other Updates

• Users can now add source type and name metadata while loading data to indicate source systems
• Users can now disable auto-generation of TDEs
• New Gradle tasks for clearing user artifacts and modules, and describing users and roles
• Users now have the ability exclude processed items when running a step

Also includes enhancements and bug fixes, for more details check release notes.

Full Release Notes

Package	Key	Method
marklogic-datahub-5.4.0.war	fd8fc0fdc5fd97d289f9aba89eccc39becb82c11	SHA1SUM
marklogic-data-hub-5.4.0-client.jar	28ec7f8061fefc4212f84f4af96f9d7c7d8f2070	SHA1SUM
marklogic-data-hub-central-5.4.0.war	f03fbe9c56cbed73153e9a18660e936367bb47e2	SHA1SUM