The seemingly endless number of patient forms and documents filled out at hospitals, doctor’s offices and clinics are a large part of the ongoing healthcare data generated today that becomes part of our permanent health record. But this is just the tip of the iceberg. Behind the scenes, each MRI procedure, pharmacy transaction, CAT scan, EKG record, insurance claim, billing record and blood analysis represents a huge amount of data that sits “below the waterline” and out of sight.

Healthcare data is generated by numerous systems and in a wide variety of formats—syslog, custom application logs, XML, HL7 and myriad other formats. Add to this business vertical an IT vendor technology landscape that is influenced by mergers, acquisitions and disparate and conflicting development processes. It’s no surprise that most healthcare applications do not conform to a single data format. With so many unique formats to contend with, managing this data and deriving value from it represents an ongoing struggle for healthcare industry IT professionals.

The breadth of this data is one facet of the challenges facing healthcare organisations; the scale of this data is massive and preventing misuse is no small undertaking.