Source-to-Target Data Mapping: All You Need to Know

Understanding how data gets ingested to its data warehouse is important for every business. It not only  helps in making forecasts, evaluations, and future decisions, but  also enables  businesses to use the same data for a variety of new products that they are working on. This requires careful data mapping where each sourced column, key, or data point is mapped to the right column, key, or data point on the destination.  In this article, we will discuss the most important aspect of data mapping i.e. source to target mapping and why it is so important for any business to work effectively.

What is Source-to-Target Mapping?

Consider this, you shift from one house to the other. Now, will you leave all the items you owned in the previous house, or will you move each one of these items to the new house exactly at the proper place? Surely you will opt for the second as you will need these items from time to time to run your house chores effectively. Similarly, a business operation requires data to run effectively. For this purpose, it needs to source data from the initial data mart to the final destination, which is commonly the data warehouse. A good example here will be of a restaurant that has a dine-in facility available with over 20 tables. It is listed on a travel website where the data is passed in real-time about the status of each table - whether it is vacant or occupied.  Now, if the sourced data (from restaurant to the site’s booking system) is not matched to the right column in the destination system, that restaurant will start to get double bookings for the same table or lose bookings due to outdated data.  This is just one instance. The same thing can occur for seats on a flight, for rooms in a hotel, or even critical issues such as the number of beds in a hospital. Botched data mapping can lead to data loss, which in turn hits the bottom line of any organization.

Why Use Source-to-Target Data Maps

  • For creating pre-ETL data models
  • When a new data point is added to the sourced system
  • Before moving data to OLAP or Tabular system for report generation
  • When a new repository is added to the data model
  • For consolidating data of multiple sources into a single data warehouse or a data lake
  • After applying new business transformations and conversion rules to sourced data

List of Examples of Source-to-target Mapping 

Here are a few examples of how source-to-target data mapping helps in each industry.

Source-to-target mapping in Retail

Source to target mapping is widely used in the retail industry. Retailers use Electronic Data Interchange (EDI) files to exchange data in-between suppliers and retail partners. These include different EDI data formats such as Product Activity Data (EDI 852), Purchase Orders (EDI 850), advanced ship notices (EDI 856), to name a few. All these files help suppliers to exchange data digitally without any issues. Since EDI formats allow standardized communications between parties, they allow saving time, costs of operations, and most importantly make suppliers respond faster to demand changes. Now the question is, how source to target mapping helps with sending and receiving data through these EDI formats. The source-to-target mapping allows the mapping of EDI file data to any common format used by the suppliers. This can be CSV, XLS, or some other file type that the suppliers can open in a program of their choice or move to a data warehouse for later evaluations.

Source-to-target mapping in Healthcare

Similarly, healthcare services have their own set of data standards designed by international health care services. These standards are HL7 for healthcare information and EDI 837 for healthcare appliances and equipment SKUs. The information needs to be ingested into the data warehouse under the HIPAA compliance guidelines regularly so that insurance companies, suppliers, and even retailers dealing in  healthcare equipment are aware of the demand and supply of the equipment.

Source-to-target mapping in Finance

Take forex trading. Each point is changed in real-time and a slight difference in time can lead to a loss of millions of dollars. The data is not stored in a central database but kept in data centers spread across the globe. The data is exchanged between data centers, of brokerage houses, and central banks, and financial firms in real-time without even a split second delay.  Now if a company is creating an app for forex trading, the sourced data must be pointed to the right node at the destination. This would allow accurate functioning without any loss. 

Source-to-target mapping in Travel

Flights are now mostly booked through online apps. Apps like Kayak, Skyscanner, Booking, and various others use data from the central flight database and show it to the user. Users can book seats of their choice on the flight and even find cheaper rates through delayed booking. To move data from the central database to the right data points in the website/application, companies require source-to-target data mapping software.

How ETL Software Make Source-to-Target Mapping Easy

ETL software like Astera Centerprise, Talend, Informatica, and various others offer data mapping features that users can use to easily map data from a source to a destination. After mapping data from a source to a destination, users can easily move data from many data sources to a data destination of their choice. This is an example of how Astera’s data mapping software can move data from a source to a destination while applying hundreds of transformations and customizations.  It allows users to create one-to-many or many-to-many data maps. Once the data maps are ready, users only have to run the ETL software to easily move data from the source to the destination within seconds. Similarly, the maps can also be scheduled to run after a set interval or even in real-time through job scheduling configurations.

Bottom-line

Businesses always want to extract the most value from the data they generate regularly. Source-to-target data mapping software has made it easier for them to easily move data from various data marts and lakes to a single data warehouse for a more holistic view of their organizational performance.  We hope that this blog has helped you understand more about why source-to-target mapping is necessary for each industry and how you can benefit from it. Learn more about source-to-target data mapping software to boost your business revenue.

Sharjeel Ashraf

Sharjeel loves to write about all things data integration, data management and ETL processes. In his free time, he is on the road or working on some cool project.

Leave a Reply

Your email address will not be published. Required fields are marked *