PeopleSoft To JD Edwards
I recently gained the opportunity to work on a migration project for one of our clients, where I was assigned the task of initiating a PeopleSoft to JD Edwards data conversion. Through this project, I learned the process of data conversion, the strategy behind it, general challenges faced, and how to identify opportunities for future migrations. In this article, we look at each aspect in detail.
The Importance of Data Conversion
For an organization, data is an asset that takes years to gather. When there is a change in the system (in this case PeopleSoft to JD Edwards), the data stored in the legacy system needs to migrate to the new system. Generally, different systems store, support, and retrieve data in their own unique way. Sometimes there are multiple sources of data, which adds complexity to the data conversion process.
A poor data conversion leads to project failure, as well as wasted effort, time and money. Many projects miss the deadline and go over budget due to poor data conversion, sometimes failing completely. In addition, the performance of the new system depends on data conversion, so this aspect is especially key in the long run.
Challenges in the Data Conversion Process
There are various challenges faced during the process. In our case, the PeopleSoft to JD Edwards data conversion process was no exception. The common challenges are:
- Identifying what data to convert
Based on the nature of the business, the team needs to identify what data needs to be migrated. However, there is some information that is essential, like customers, vendors, company configurations, business entities, financial, assets, etc. Sometimes historical data and transaction data also need to be migrated.
- What the mapping should be
Once the data is identified, the next process is finding the place to store it in the new system. Business analysts map this data to the new system’s table fields based on the module.
- Working with multiple teams
Generally, teams have very limited access to the legacy system, and must work with their client’s IT team to get information on system tables and field definition.
- Engaging end users
A client’s end users work on the legacy system, and provide great help in identifying the fields definition and further information. These users are generally working from different locations and time zones, which effects the data conversion team. Their availability is also challenging due to their day-to-day job tasks, making coordination very difficult.
- Dealing with multiple data sources
The legacy system may have multiple integrations with other systems, sometimes using data from other 3rd party source. In our engagement, the vendor’s data was maintained in two places, which was within PeopleSoft and a 3rd party system. Both of which needed to be migrated to JDE.
- Huge data volume
The transaction data grows over time and takes significant time to migrate. Therefore, data is converted in phases, giving new responsibility to the data conversion team.
- System limitations
Most systems use their own field definitions (i.e, a string field storing 40 characters in a legacy system). However, the new system does not have a field to store this data in a respective table. In such cases, customization is needed to avoid data loss tag tables.
- Dealing with dynamic data
We know business keeps going during the migration process. As transactions in the system keep growing, updates to the data occur frequently within the legacy system. The data conversion team should consider this before switching over to the new system.
- Considering the best tool/technology
Once data and data mappings are identified, it is now time to introduce new tools in the system. Various tools and technologies are available to choose, and picking the right one is crucial. For this engagement, we considered the development of JDE reports, as it is easily controlled and good for handling huge data volumes.
The Data Conversion Strategy
As previously discussed, our team studied the business process and came up with the list of data sets that need to be migrated. These included data for vendors, customers, franchisees, open balances, open payments, branch plants, and assets. We decided to use a standard JDE-Z process to migrate data to main tables. For this, some custom programs were developed.
First, data is extracted in pipe delimited test files. Then, custom programs read these files and dump this raw data in custom JDE tables. These custom tables work as a staging place for the data, which also gives way to check and verify results. Once all data is loaded to the staging area, custom JDE Universal Batch Engines (UBEs), massage the data based on the rules defined, and load the records into the respective JDE-Z tables.
Business analyst have the opportunity to verify how the records were created in the new system before finalizing. Here, programs use transformation rules for User Defined Codes (UDCs), as the JDE system use UDCs that are different from PeopleSoft. In this process, a custom application was created where the module team could configure rules independently. This gave flexibly to set up new transformation rules, and modify existing rules as need. This application is a central place for all transformation rules across different conversion programs.
This strategy was used for the following reasons:
The team was not dependent on others for development issues (if another technology was used).
Conversion programs can be run by business analysts without the involvement of other teams.
In the event of a change or new discovery, change to JDE custom programs are easy to debug and incorporate.
Once data is extracted, the conversion team independently worked on the data.
As all development is done within JDE, the team has more control on programs.
Standard Z-processors have the option to run the program in proof mode.
The approach is time saving and cost effective.
This project was also an opportunity to fix various data related issues in the legacy system. As this data was accumulated over the years (and by various processes), there were many inconsistencies identified. These are as follows:
- Inconsistent format of data (e.g., phone number format is not consistent throughout the system).
- Duplicate information (e.g, one vendor had multiple contacts, and many of these were entered multiple times).
- In the legacy system, there were many orphan records (e.g, there are contact records present without customer or vendor record).
- Archival of old data present.
- Previous cleanup of wrong data with fixed inconsistencies.
The Importance of Data Conversion
There were six User Acceptance Tests (UTAs) that were planned for the project. There were changes expected, but all were fixed quickly because of the robust design and use of JD Edwards development technology. No major changes/issues were identified in the converted data as well. Altogether, the project was delivered on time and within budget.
In short, data conversion is a very important aspect in many ERP implementation projects. Further, projects can be delayed and go over budget because of poor data conversion. This process should be planned in the early phase of the project to ensure success. It is key to keep the data conversion program (technology) simple, as these are run only once, just before go-live. Once you’ve converted all required data, these programs are no longer needed, so do not exhaust too much effort on the program itself. Rather, focus on the overall design so the converted data is clean and complete.
There’s more to explore at Smartbridge.com!
Sign up to be notified when we publish articles, news, videos and more!