PeopleSoft To JD Edwards
Data Conversion

The Importance of Data Conversion

For an organization, data is an asset that takes years to gather. When there is a change in the system (in this case PeopleSoft to JD Edwards), the data stored in the legacy system needs to migrate to the new system. Generally, different systems store, support, and retrieve data in their own unique way. Sometimes there are multiple sources of data, which adds complexity to the data conversion process.

A poor data conversion leads to project failure, as well as wasted effort, time and money. Many projects miss the deadline and go over budget due to poor data conversion, sometimes failing completely. In addition, the performance of the new system depends on data conversion, so this aspect is especially key in the long run.

Challenges in the Data Conversion Process

There are various challenges faced during the process. In our case, the PeopleSoft to JD Edwards data conversion process was no exception. The common challenges are:

• Identifying what data to convert

  Based on the nature of the business, the team needs to identify what data needs to be migrated. However, there is some information that is essential, like customers, vendors, company configurations, business entities, financial, assets, etc. Sometimes historical data and transaction data also need to be migrated.

• What the mapping should be

  Once the data is identified, the next process is finding the place to store it in the new system. Business analysts map this data to the new system’s table fields based on the module.

• Working with multiple teams

  Generally, teams have very limited access to the legacy system, and must work with their client’s IT team to get information on system tables and field definition.

• Engaging end users

  A client’s end users work on the legacy system, and provide great help in identifying the fields definition and further information. These users are generally working from different locations and time zones, which effects the data conversion team. Their availability is also challenging due to their day-to-day job tasks, making coordination very difficult.

• Dealing with multiple data sources

  The legacy system may have multiple integrations with other systems, sometimes using data from other 3rd party source. In our engagement, the vendor’s data was maintained in two places, which was within PeopleSoft and a 3rd party system. Both of which needed to be migrated to JDE.

• Huge data volume

  The transaction data grows over time and takes significant time to migrate. Therefore, data is converted in phases, giving new responsibility to the data conversion team.

• System limitations

  Most systems use their own field definitions (i.e, a string field storing 40 characters in a legacy system). However, the new system does not have a field to store this data in a respective table. In such cases, customization is needed to avoid data loss tag tables.

• Dealing with dynamic data

  We know business keeps going during the migration process. As transactions in the system keep growing, updates to the data occur frequently within the legacy system. The data conversion team should consider this before switching over to the new system.

• Considering the best tool/technology 

  Once data and data mappings are identified, it is now time to introduce new tools in the system. Various tools and technologies are available to choose, and picking the right one is crucial. For this engagement, we considered the development of JDE reports, as it is easily controlled and good for handling huge data volumes.

The Data Conversion Strategy

As previously discussed, our team studied the business process and came up with the list of data sets that need to be migrated. These included data for vendors, customers, franchisees, open balances, open payments, branch plants, and assets. We decided to use a standard JDE-Z process to migrate data to main tables. For this, some custom programs were developed.

First, data is extracted in pipe delimited test files. Then, custom programs read these files and dump this raw data in custom JDE tables. These custom tables work as a staging place for the data, which also gives way to check and verify results. Once all data is loaded to the staging area, custom JDE Universal Batch Engines (UBEs), massage the data based on the rules defined, and load the records into the respective JDE-Z tables.

Business analyst have the opportunity to verify how the records were created in the new system before finalizing. Here, programs use transformation rules for User Defined Codes (UDCs), as the JDE system use UDCs that are different from PeopleSoft. In this process, a custom application was created where the module team could configure rules independently. This gave flexibly to set up new transformation rules, and modify existing rules as need. This application is a central place for all transformation rules across different conversion programs.

This strategy was used for the following reasons:

Opportunities Identified

This project was also an opportunity to fix various data related issues in the legacy system. As this data was accumulated over the years (and by various processes), there were many inconsistencies identified. These are as follows:

• Inconsistent format of data (e.g., phone number format is not consistent throughout the system).

• Duplicate information (e.g, one vendor had multiple contacts, and many of these were entered multiple times).

• In the legacy system, there were many orphan records (e.g, there are contact records present without customer or vendor record).

• Archival of old data present.

• Previous cleanup of wrong data with fixed inconsistencies.

The Importance of Data Conversion

There were six User Acceptance Tests (UTAs) that were planned for the project. There were changes expected, but all were fixed quickly because of the robust design and use of JD Edwards development technology. No major changes/issues were identified in the converted data as well. Altogether, the project was delivered on time and within budget.

In short, data conversion is a very important aspect in many ERP implementation projects. Further, projects can be delayed and go over budget because of poor data conversion. This process should be planned in the early phase of the project to ensure success. It is key to keep the data conversion program (technology) simple, as these are run only once, just before go-live. Once you’ve converted all required data, these programs are no longer needed, so do not exhaust too much effort on the program itself. Rather, focus on the overall design so the converted data is clean and complete.

Keep Reading: Smart Fields on JD Edwards Custom Reports