The Bulk Upsert Mongo block enables bulk upsert operations in MongoDB. Upsert is a database operation that combines update and insert actions, allowing new data to be inserted while updating existing records in bulk. Use this block when handling more than ten queries simultaneously.

Example Scenario

Requirement

A retail company tracks customers who achieve a specific target within a given timeframe. These customers qualify for rewards as part of the company's loyalty management program, which Capillary handles. The data is exported from the company's database as a CSV file and sent to the Capillary system. If a record exists, Capillary must update the record available in the Capillary database; otherwise, the system should create a new record.

Solution

To insert and update a large amount of data in MongoDB, create a dataflow using the Bulk Upsert Mongo block. This block updates existing data and inserts new data if it is missing in the specified collection.

Refer to this example dataflow to understand how it is configured for a use case.
Make sure you have access to DocDemo org (100737) and access to Neo.

Configuring the Bulk Upsert Mongo Block

To configure the Bulk Upsert Mongo block,

1. From the dataflow canvas, click on the node and select the Bulk Upsert Mongo block.
2. In the Block Name, enter the name of the block.
   Note: Block names cannot contain spaces or special characters, except for underscores (_). Use camelCase or snake_case formatting.
3. In the Collection Name field, enter the collection name to be updated.
4. In the Mode field, choose the mode of the update as bulkupsert.
5. In the Query field, enter the data to ingest into MongoDB.
   Note:

• Both a filter and a query are mandatory in each object.
• Each object contains:
  • Selection filter – Finds specific records in the database.
  • Query – Defines how the record should be updated.
• DAO methods interact with the database and execute queries.
• Expression handling enables dynamic query generation, allowing logic or conditions to modify queries before execution.
• If the filter or query is missing, the operation fails due to an invalid input query.
  
  
  Below is a sample query built in the Script block.
  Code in the Script block

  const data = getApiRequest().body;
  const upsertData = data.map(doc => ({
      query: JSON.stringify({$set: doc}),
      filter: JSON.stringify({"namespace": doc.nameSpace}),
  }));
  

6. (Optional) In the Options field, specify any additional options for the query. These options modify query behavior and how results are returned.
   Note: For information on available options, refer to the MongoDB documentation on options.
7. Configure the Input execution logic, Cachable feature and define the execution path as per the requirement.
8. Click Done.