v4.2.0

Release Notes for DHF 4.2.0

Click Here to See the Release Notes:
https://marklogic.github.io/marklogic-data-hub/release-notes/

v4.1.1

Release Notes for DHF 4.1.1

This is a bug fix and maintenance release

• Ability to pass in more than one key/value to MLCP Input Flow Options #1097, #1571
• Better handling of nested objects as properties when property is not defined as a formal entity, array, or scalar value #1652
• hubinit now creates a stub gradle-local.properties #1736
• Flexrep config should now deploy for data-hub-FINAL #1724
• You can call your mapping 'mapping' now (v4.1.0) #1710
• You call your input flow "input" now (v4.1.0) #1709
• You call your harmonize flow "harmonize" now (v4.1.0) #1708
• Server namespaces now supported in your *-server.json config files #1631
• Trace errors now output in a much more human readable format #1402
• Several tutorial and documentation typos were fixed.

For more information on the bug fixes in this release, see the change log.

v4.1.0

Release Notes for DHF 4.1.0

The DHF project now aligns more closely with standard ml-gradle applications.

• The directory structure and locations of configuration files remain the same as in DHF 4.0.x.

• See Project Directory Structure for information on the ml-user-config and hub-internal-config directories.

• See Gradle tasks for information about ml-gradle tasks specific to DHF.

• Java deployment commands specific to the data hub were changed as follows:

  • Added:

    • DeployHubOtherServersCommand.java
    • HubDeployDatabaseCommandFactory.java
    • LoadHubSchemasCommand.java

  • Renamed:

    • from LoadUserStagingModulesCommand.java to LoadUserModulesCommand.java

  • Removed:

    • DeployHubDatabasesCommand.java
    • DeployHubFinalSchemasDatabaseCommand.java
    • DeployHubFinalTriggersDatabaseCommand.java
    • DeployHubOtherDatabasesCommand.java
    • DeployHubRolesCommand.java
    • DeployHubSchemasDatabaseCommand.java
    • DeployHubServersCommand.java
    • DeployHubStagingSchemasDatabaseCommand.java
    • DeployHubStagingTriggersDatabaseCommand.java
    • DeployHubTriggersDatabaseCommand.java
    • DeployHubUsersCommand.java
    • DeployUserRolesCommand.java
    • DeployUserServersCommand.java
    • DeployUserUsersCommand.java

• Custom triggers must now be added to ml-config/databases/databasename/triggers. (In previous releases, triggers were added to hub-internal-config/triggers or ml-config/triggers.) See ml-grade Project Layout for more information on triggers.

DHF Java API changed to allow DHF to use the Spring Framework.

• The Java HubConfig class now has a singleton scope.
• See DHF Java Library for more information.

---

For more information on the bug fixes in this release, see the change log.

v4.0.3

This is a maintenance release with just bug fixes.

For more information on the bug fixes in this release, see the change log: https://github.com/marklogic/marklogic-data-hub/blob/master/CHANGELOG.md.

v4.0.2

This is a maintenance release with just bug fixes.

For more information on the bug fixes in this release, see the change log: https://github.com/marklogic/marklogic-data-hub/blob/master/CHANGELOG.md.

v4.0.1

This release prepares DHF for the environment of our Cloud-based Data Hub Service project, mainly by refactoring tests and deployment.

This is a minor release with just bugfixes, and testing changes.

See changelog https://github.com/marklogic/marklogic-data-hub/blob/master/CHANGELOG.md for information on bugs fixed

v4.0.0

This release takes care of many bugfixes and issues identified in previous versions of DHF, in addition to delivering new capabilities to the Operational Data Hub.

• Security Model and Fixes
• Project structure compatibility with ml-gradle
• Installation of enode code from DHF jar
• Declarative mapping of harmonization data flows, to automate basic steps in a Data Integration pipeline.
• Sub-document identification and securing of Personally Identifiable Information.

Important Changes

1. Security model update. DHF 4.0.0 has documented and tested its security model. Depending on how your projects were configured, you may have to revisit how your flows are configured, deployed, and run.
2. For new projects created with DHF 4.0.0 there is no longer a TRACING database. The functions of jobs and traces have been merged to utilize a single database. Projects that are upgraded from DHF 3.0.0 will no longer use or deploy to their former TRACING database. The upgrade process does NOT remove the database, though.
3. New projects created with DHF are compatible structure-wise with ml-gradle
4. the MarkLogic XQuery and JavaScript code in DHF is now deployed as part of the install process, rather than shipped with MarkLogic Server.

Since 4.0.0 is a major release and there are some architectural changes, please refer to https://marklogic.github.io/marklogic-data-hub/docs/release-notes/ for more information on upgrading and what has changed.

V2.0.5

Maintenance Release

This is a bug fix focused release that addresses the issue of streaming large batches of URIs from the collector via the DHF to be memory efficient and properly queue the stream on disk. Please upgrade to this version if you harmonize against large batches.

Change Log

v2.0.5 (2018-08-10)

Full Changelog

Fixed bugs:

• Huge URI/ID response from collector now properly streams with efficient memory use and excess queues on disk in system tmp

Closed issues:

• Issue when using a collector on a huge amount of URIs #1201

V2.0.6

Maintenance Release

This is a bug fix focused release that addresses an issue whereby DHF can enter a condition where the output of harmonized flows appears as 'canceled', despite it successfully completing.

Change Log

v2.0.6 (2018-08-10)

Full Changelog

Fixed bugs:

• DHF can enter a condition were harmonize flows return a status of canceled, even if they succeed. This release corrects that issue.

Closed issues:

• Harmonize flow finishes with "CANCELED" status even though successful #1259

v3.0.0

DHF 3.0.0 Major Version Release!

THIS VERSION (3.x) WILL NO LONGER BE ACTIVELY MAINTAINED OR DEVELOPED AS OF AUGUST 8th 2020.

PLEASE REVIEW the DHF website AS 3.0 INTRODUCES BREAKING CHANGES TO 2.X APPLICATIONS. YOU WILL NEED TO UPDATE YOUR CODE AND UPGRADE YOUR FLOWS.

Change Log

v3.0.0 (2018-05-03)

Full Changelog

Implemented enhancements:

• Update Spring Batch example to version 1.4.0 #622
• Update writers to be batched vs individualized - delete/dupe #617
• Quickstart data hub job status/error popup needs word wrap #582
• Export traces #571
• Create gradle command to generate a TDE Template #551
• Allow specifying flow options for harmonization flows run from quickstart #504
• Migration guidance from 1.0 (8) to 2.0 (9) #373

Fixed bugs:

• Loading documents through input flow is failing #883
• gradlew quick-start:e2eLaunch could not find or load main class com.marklogic.quickstart.Application #877
• Tutorial: version numbers don't match #852
• Example in documentation is incorrect #845
• README.md link to "Data Hub Framework website" in "Advanced Hub Usage" is broken #837
• defaults are missing on hub init tab #833
• Getting Started Tutorial: More issues #818
• Running input flow produces error "MISSING_CURRENT_TRACE" and the documents are not loaded #814
• Missing dhf.sjs when creating a new flow (blocker) #813
• Upgrade npmVersion to 5.6.0 on build.gradle to avoid error on Windows #804
• Tutorial link hard coded to old (2.0.3) release #801
• Reverse DB EndToEndFlowTests fail on AWS #800
• quickstart harmonize flow view elides tab labels to meaninglessness #798
• QuickStart Browse Data throws XDMP-LEXVAL #797
• Documents are not shown under Browse Data STAGING database after running the input flow #788
• Remove mimetypes from gradle config #780
• Install screen on quickstart is broken, unable to install hub [blocker] #778
• Run undeploy tasks with configured mlManageUsername #766
• Error when running gradlew mlUndeploy #762
• MLCPBean doesn't have -ssl option #760
• mlDeploy fails when run by an LDAP user with full admin rights #759
• QuickStart won't connect to HTTPS-enabled App Services #752
• Search results on jobs page is showing the wrong results when searching for "input" jobs #746
• Trace view is not displayed after clicking the trace link #738
• One entity's indexes configurations clobber all the others' #711
• When modeling Order entity, needs to add element range index on "id" property #705
• Quickstart tutorial doesn't have $version on content.sjs, but the screenshots have it #701
• QuickStart harmonize flow settings not persisted during browser session #684
• Primary key is not retained if it's clicked first when adding a property #679
• Huge ID lists from a collector fail with FRAGTOOLARGE if Tracing is on #658
• Old entity name is still retained on property entities type #651
• run-flow rest extension is not setting a default job-id #620
• Tutorial - Can not create "Harmonize Orders" flow #602
• Collector's disk queue is filling up #601
• QuickStart Project Initialize does not recognize change to hostname #585
• Object type changed after upgrade to Hub 2.0 so fields are missing or undefined #583
• The mlUndeploy task does not completely remove a data hub #580
• MLCP failing on INPUT FLOW after upgrade #579
• info block for JSON #577
• QuickStart - Browse Data does not display content for certain URIs #557
• Add Spring-Batch jobRepo configuration to data-hub-JOBS database & app-server? #493
• Added Price, but didn't see it in the Product entity #408
• A trace is created with an invalid format #397
• Hub (un)install time on windows is horrible #253
• Add --disable-host-check to allow external access #773 (wooldridge)