Then there is handling missing data, managing the outliers, fixing structural errors, removing doubled up data, aligning calendars, and many more!
The Curse of the In-House Analyst
Often the job of pulling retailer data from portals, or receiving EDI files from third-party providers, sits with an in-house analyst. It is a time consuming and often painful process to collate data.
In any given week, there is a high probability of problems with one or more files, resulting in a cascading issue. Incorrect or incomplete data is loaded into the company’s database before being pushed out to the users of the data. The analyst is charged with figuring out which data is incorrect; whether the raw file was correct but loaded incorrectly, or whether the raw data was wrong in the first place.
Assuming that can be resolved, there is then a process of elimination, working back up the chain to determine where the broken link is. Requests go out for replacement files. Data has to be deleted from the database; notifications have to be sent out to users. Once replacement data is sourced and loaded, reports can be re-run and distributed.
Users are frustrated, IT is frustrated, Supply Chains are screaming for updates, planners are stuck, and nobody wants to take a call from the buyer! Add in multiple regions and timezones and the need to collate data across divisions, and you have a recipe for stress!
Analysts Prefer Analysing Over Data-Cleansing
Yet analysts are hired to analyse. Their job is to data-mine; to find the key insights quickly to support smarter, faster decision making. Allocating the right products to the right stores; interpreting the results of the trade promotion; providing insights to the product development team. As channel boundaries become blurred, consumer purchasing behaviour shifts, and as demand for instant gratification grows, the need to focus on data outputs rather than data inputs becomes ever more critical.
We understand the challenges of collating Electronic Point Of Sale data from multiple retailers in different formats, because that’s what we do every week, with files from over 200 retailers across the globe. We have unrivalled experience in gathering, collating, cleaning and normalizing data, so that you can be confident in working with data you can trust.
Furthermore, we merge the retail data with Master data such as Calendars, Store Masters and Product Masters, to facilitate analysing the data using common attributes.
So let us wrangle your data, and free up your analysts to focus on what they’re best at – analysing!
For more on our data cleansing services, or to discuss your specific data needs, talk to us by emailing