Dell Boomi Document Data Manipulation Techniques

Dell Boomi Document Data Manipulation

Dell Boomi (an independent business unit of Dell) is an Enterprise Integration Platform (EIP) that transforms “unconnected” companies into connected businesses that can engage and run everywhere. Their platform provides easy drag-and-drop integration between an ever-growing network of software-as-a-service, on-premises applications, and data repositories.

Integration requirements across all these systems are fairly complex. We had to apply several Dell Boomi data manipulation techniques to transform source data flowing through different steps in the processes. A “document” in this case is a set of data flowing through the Boomi process itself. A document can be a single record, a group of records, an Electronic Data Interchange (EDI) transaction or an entire file.

This blog describes some of the techniques applied while processing documents during an integration phase. We’ll break these down in a “what, where, and example scenario” fashion – what exactly the technique is, where it can be applied, and the scenario it serves to support.

Smartbridge is a Dell Boomi Partner

Breaking Down the Techniques

Splitting Documents

• What: Splitting a document into multiple documents – a document can be split by a profile item or by line using Data Process shape.

• Where: This technique can be applied in situations where each line item or profile item needs to be validated (or processed) on a per-record basis.

• Scenario: A Sales Order document received from Salesforce contains multiple Sales Order lines, and you need to process each line independently.

Combining Documents

• What: Multiple documents can be combined into a single document using Data Process shape. This option appends each document’s data to the previous one, creating one combined document.

• Where: This technique can be applied in situations where common data between multiple documents being processed needs to be combined for further processing.

• Scenario: You have a requirement to query the accounts associated with multiple Sales Order documents. Use this technique to combine all the documents together and process them to determine the accounts.

Getting Unique Values

• What: From the common data in multiple documents, get unique values. There is no process shape which can give this functionality an out-of-the-box appeal. However, with the combination of Data Process & Map Shapes, the desired functionality is accomplished.

• Where: This applies in situations where one is referencing common data from multiple documents. If it is necessary to get any additional information from external systems, this technique can also be applied to reduce the number of calls to the external system.

• Scenario: In continuation with the previous technique, use this technique to get unique accounts from multiple Sales Order documents. These unique values can then be used to get additional account details from external systems.

Joining Documents

• What: Similar to joining data between database tables on a key column, more than one document can be joined to derive a new one, which includes data from all these documents.

• Where: This technique can be applied in situations where it’s necessary to get data from other documents by matching key columns. With the combination of Map & Document, Data Cache desired functionality can be achieved.

• Scenario: Consider the scenario where a Sales Order document has an account ID, and it’s necessary to get the account details such as a name and address. Desired functionality in Document Data Cache and Map Shapes can be achieved.

Manipulating Content

• What: Manipulate document data using Data Process shape.

• Where: To manipulate the document’s contents and perform special processing requirements, document specific strings can be replaced, or custom script can be used for any advanced special requirements. For more information about this technique, visit my previous blog post.

• Scenario: Converting document data into an HTML-friendly format using search/replace techniques. It can also be used for sending HTML friendly email notifications.

These are just a few techniques that can be used to meet data integration requirements. As the data flows in the form of a “document” between the process steps, several more techniques can be applied to manipulate the document’s contents to convert to a different format or type.

Keep Reading: Format Document Data to HTML in 3 Steps with Dell Boomi