Analysis in the Age of Big Analog Data

 Read in  |   Print

"Within one year of developing and implementing this solution, we estimate that we now analyze up to 95 percent of our data and have reduced our test cost and number of annual tests because we do not have to rerun tests."

- Simon Foster, Jaguar Land Rover

The Challenge:
Implementing a comprehensive data management solution to manage and analyze up to 500 GB of time-series data per day, generated by over 200 data loggers continuously collecting data and ad-hoc measurements performed by over 400 engineers.

The Solution:
Building a system based on DIAdem software and DataFinder Server Edition to index the metadata of any file, regardless of its origin, and create a workflow to search, inspect, analyze, and report on data 20 times faster than any previous manual method.

Author(s):
Simon Foster - Jaguar Land Rover
Pablo Abad - Jaguar Land Rover

Jaguar Land Rover (JLR) is the flagship for two iconic British car brands known for rugged design and luxury. To ensure the quality and reliability our customers demand, we place a strong emphasis on advanced design, engineering, and technology—an emphasis that has driven us to invest more in research and development than any other manufacturing company in the United Kingdom.

For the more than 400 engineers who work in our powertrain calibration and controls department, this investment includes implementing new strategies and solutions from NI to better capture and manage huge volumes of raw test data to make smarter decisions before a vehicle ever goes to market.

With better access to better data, we can make informed decisions when it comes to building superior automobiles, enhancing JLR’s reputation, and most importantly, providing our customers with a product that lives up to our high standards.

Faced with managing up to 500 GB of time-series data collected daily, we found we often repeated tests because we could not properly locate specific results. We based our original analysis routine on a manual process. The team estimated it took 20 times longer than our current solution, which is based on a fully automated analysis routine. We traditionally used multiple analysis tools, all of which required special scripting to implement algorithms, so nothing was standardized, including metadata and channel names.

This process led us to analyze only 10 percent of the data we captured when testing our vehicles. Only understanding a fraction of one’s test data means running the risk of missing out on key information, which can lead to inefficiency in the design process and costly delays.

Figure 1: A manual process allowed for only 10 percent of data to be analyzed.

Technology

To address this Big Analog Data challenge, we benchmarked nine tools to determine which platform would be the best for our application. System requirements included the ability to automate uploads, combine metadata from multiple sources, add metadata that was not originally saved and search files from metadata. We also needed an interactive tool that empowered our engineers to perform their own pre-defined routines or custom ad-hoc analysis. We had to be able to run the analysis on select files or as a batch process, and reporting templates needed to produce consistent, reliable reports to aid in decision-making. Finally, the platform had to integrate into our existing data acquisition processes with the ability expand to other JLR departments in the future.

After reviewing these nine tools, we chose to build our solution on DIAdem software and DataFinder Server Edition for a variety of reasons, in addition to the above criteria. One key reason is that DataFinder Server Edition can index metadata to which the team can send queries to find specific test results. With DIAdem, we have peace of mind knowing that we have the option to load over 1,000 file formats, so if our data acquisition processes change, we do not have to worry about compatibility with our data analysis program. We can also interactively create analytics dashboards without programming and selectively load data from multiple files.

When testing our vehicles, we collect data in a variety of ways from a variety of data acquisition devices. Whether it’s through data loggers capturing information from sensors during a test drive or by directly connecting laptops to the vehicle to collect network data–CAN, MOST, FlexRay, or ECU protocols–CCP, XCP, ETK, all data that’s captured is automatically transferred to a data analysis process where it is checked for proper metadata. We cross-reference metadata from several sources and calculate any missing parameters (such as average temperature, speed, MPG, and more). We then save these in a server based on DataFinder Server Edition.

At this point, any JLR engineer can query the data and perform an analysis routine on all test results that meet the specified parameters. Different types of analysis can be applied to the data. An engineer can choose to run a batch process, sometimes on thousands of files, using a predefined analysis routine or can create an ad-hoc analysis routine on data that requires closer inspection. The result of any analysis routine is a templated report to help us make data-driven decisions faster.

Within one year of developing and implementing this solution, we estimate that we now analyze up to 95 percent of our data and have reduced our test cost and number of annual tests because we do not have to rerun tests.

Figure 2: An automated system using DIAdem software and DataFinder Server Edition enabled us to analyze up to 95 percent of the data we collected

Thanks to these benefits and more, we now find and address more issues before passing final products to the customer. As a result, customer satisfaction ratings are higher than before because the products are more robust.

Author Information:
Simon Foster
Jaguar Land Rover
sfoste31@jaguarlandrover.com

Bookmark and Share


Explore the NI Developer Community

Discover and collaborate on the latest example code and tutorials with a worldwide community of engineers and scientists.

‌Check‌ out‌ the‌ NI‌ Community


Who is National Instruments?

National Instruments provides a graphical system design platform for test, control, and embedded design applications that is transforming the way engineers and scientists design, prototype, and deploy systems.

‌Learn‌ more‌ about‌ NI