Process Data Management: To Excel or not?

  • 14th October, 2013

Microsoft Excel is ubiquitous in any organization. My current context of "any organization" is a biopharmaceutical organization involved in development and manufacturing of biotherapeutics. We all use Excel in our day-to-day activities for storing data and analysis. Most of it is for personal consumption and sometimes to share information with our colleagues. Excel spreadsheets offer both speed and flexibility that no other software can offer. Speed for setting up the tables and columns in which we wish to capture data and flexibility for the way we wish to organize these to record data. Popularity of Excel spreadsheets are primarily due to

  • Its WYSIWYG (what you see is what you get - always) which is not the case in databases
  • Capture is same as data output
  • No involvement of IT folks while setting up capture records which is tremendously faster and less annoying for scientists and engineers than to deal with techies
  • It has personal flavor
  • And its free as it comes bundled with Microsoft Office (the lifeline of enterprise productivity). We do not need to go through long and tiring budget approval battles for a new data system.

These personal data records (aka spreadsheets) can however proliferate fast and become a nightmare to manage which many database and software companies (including ours) term as "spreadsheet madness or mania". Here are a few things that does not go in Excel's favor:

  • Sharing: Ability to share data in real-time so that all my employees can have access to same/single source of truth
  • Data Duplication and data authenticity: Same Information is maintained individually by many employees in his/her way own way.
  • Storing Time Series data
  • Performing Advanced queries
  • Real-time integrated data access: to visualize and trend process data in a holistic way (unified process information from all process steps both historical and current)

If your process execution recording system (experimental run records or batch production records) is still on paper we are of the view that good old Excel still can still play a very cost effective and useful role in developing a world class process data management system and should not be shunned away just yet. Excel makes an excellent choice if used just as a templated data recording tool owing to the following:

  • Employee familiarity with the tool leading to extremely low learning curve and resistance in adoption
  • Recording templates can be maintained and distributed by scientists and engineers themselves without the need of IT intervention
  • Almost zero implementation time (less than 5-6 business days)

The other tasks of data storage, process aggregation and data analysis should be outsourced to advanced tools that involves databases, historians and data visualization/analysis platforms. One such solution can be using Simplyfeye's ProcessPad platform as shown below: