Remove Duplicates

Node catalog Transformations

On this page

Delete items with matching field values

Usage and specifics

compareFields sets comparison fields. excludePreviousRuns extends checks to previous runs: this differs from deduplicating only the current batch.

Technical type : remove_duplicates. Runtime : nodulz.

Credentials and prerequisites

This node defines no dedicated external credential. Remote sources or execution targets may still need their own access credentials.

See the credentials guide if a remote source or execution target requires access.

Parameters

Names below are technical keys. Defaults are not a ready-to-run configuration: complete fields required by your operation. Paths containing [] describe list items; some fields depend on the selected mode.

Parameter	Type	Default	Values / limits
`source`	`string`	`""`	-
`compareFields`	`array`	`[]`	-
`excludePreviousRuns`	`boolean`	`false`	-

Initial configuration

Initial catalog configuration example, to adapt before running. Keep secrets in credentials.

json

{
  "source": "",
  "compareFields": [],
  "excludePreviousRuns": false
}

Verification

Test with representative data. Check node output in the inspector and logs, then use editor mapping for downstream fields. A trigger test with a simulated payload does not validate external service delivery.