With ongoing conflict in eastern Europe and strikes spreading across the rest of the continent, supply chain disruption has become the new norm. This uncertainty is placing mounting pressure on UK manufacturers to review and accurately forecast their capacity, as well as to find ways to minimise this very real risk to production. Yet, despite this, October 2022 research from IDC shows that 46% of manufacturers lack the appropriate visibility to see and respond to changes in their supply chains.

It's not just these macro, geopolitical challenges that manufacturers have to contend with. Many face internal structural challenges that impact their ability to manage their supply chains.

It’s common for different departments to keep their planning processes separate from each other. This means valuable data sets aren’t shared, which can impede a manufacturer’s capacity to see the full picture of data insights and trends. This siloed approach to data management is a hangover from the era of on-premises data storage and operations and was not designed for today’s digital world. Manufacturers that are still reliant on these legacy systems and processes may find that their supply chain decision-making is based on incomplete or inaccurate information, which could be hitting their bottom line.

To rectify this, focus should shift to integrating data sets and breaking down silos.

Mark Van de Wiel, Field CTO at Fivetran

Integrating manufacturing data sources for forecasting

A good first step for manufacturing companies eager to gain rich supply chain-related data intelligence is to dismantle dependencies on on-premises data warehouses and data lakes.

By keeping data in these rigid environments, key insights are invisible to many other systems, departments and stakeholders that could benefit from them. What’s more, data warehouses and lakes require manufacturers to make significant time and financial investment, and can limit capacity for  innovation. However, when  data is centralised in the cloud, companies have access to an environment that is perfectly scalable through a pay-as-you-go scheme - making scaling as simple as clicking a button. Cloud environments also support greater options for analytical services, so manufacturers are positioned to experiment with options likely to bring the most value. 

An ongoing reliance on on-premises data limits manufacturers’ abilities to part-level forecast – a process in which data from multiple sources including warehouse inventory, historical production volumes, maintenance records and causal insights, is used to understand and predict part-level demand. Without accuracy here, manufacturers struggle to calculate subsequent revenues. Part-level forecasting informs ERP systems and ensures that manufacturers aren’t wasting time and investment on supplying too much of one resource or underestimating need and demand. Without accurate forecasts, it’s impossible to achieve the supply chain agility needed to keep abreast of disruptions and react before operations are negatively impacted.

To achieve accurate results, part-level forecasting requires high-value data taken from as many sources as possible, as soon as possible. To do this successfully in the cloud, many manufacturers have adopted an advanced replication approach so as not to slow down operations. Data replication allows for consolidated access to operational data for real-time analytics, data streaming and machine learning use cases. To achieve this, companies are increasingly using log-based Change Data Capture (CDC) as a data management strategy that continuously transfers data as it changes, with end-to-end latency reduced to minutes if not seconds.

Some alternatives to real-time log-based CDC – which are still relied upon by many manufacturers – include batch processing and snapshots. However, these strategies can add significant overhead to database systems. For example, if a company doesn’t need access to its data on a regular schedule, such as outside of working hours, then it may run batch processes or take snapshots when the load is low. Yet this isn’t congruent with today’s 24/7 world where resilience and real-time insight is a must, where even yesterday’s information could be useless for ERP processes or forecasting. Analyses based on snapshots that cannot be altered – or batch processes that are only run when data input is low – can set manufacturers back.

In contrast, real-time log-based CDC only adds the smallest load to the primary source database system meaning that data can be transferred asynchronously as and when changes occur, even when systems are on standby.

Cloud as a cornerstone for Troy

And this technology is not the product of a distant future. Manufacturers have already begun to take the necessary steps towards data integration so that options such as CDC become realistic. One example of a company which has successfully made this transition is Troy, the UK-based buying group for engineering and industrial supply businesses. Troy realised its reliance on paper-based supply chain management processes was preventing it from making informed, up-to-the-minute decisions. It decided to move its data infrastructure to the cloud, allowing it to connect multiple systems at once and improve end-to-end supply chain visibility through enhanced data availability. Not only has this proved cost-efficient, Troy is now also able to generate clear and precise reports in a range of formats including: Power BI dashboards, downloadable reports and Excel spreadsheets. This has made valuable data insights widely visible to a wider cohort of decision makers.

Troy’s use case is by no means unusual. Many companies have historically relied on what are now outdated, time-consuming approaches to data management.  As a consequence, they struggle to keep pace with the rate of change in their supply chain, and could be exposed to disruption. By moving data to the cloud and committing to keeping it integrated, manufacturers can harness an array of modern options such as CDC, to tie into and improve fundamental forecasting.

When manufacturers have faster access to all their data – regardless of its source – they can scale and adapt at the same speed as they make decisions. Log-based CDC provides manufacturers with a system that is non-intrusive and low impact on transaction processing databases, while also delivering real-time intel.

With 2023 beginning to unfold, manufacturers would be wise to implement modern data-driven supply chain strategies that are agile enough to respond to future trends.