The Laura P. and Leland K. Whittier Virtual PICU

Learn more about the VPICU

The Laura P. and Leland K. Whittier Virtual Pediatric Intensive Care Unit (VPICU)


L.K. Whittier Foundation

Supported by the incredible generosity and foresight of the L.K. Whittier Foundation since 1998, the Laura P. and Leland K. Whittier Virtual Pediatric Intensive Care Unit (VPICU) has brought the benefits of information science technology to the care of critically ill children across the country. In partnership with L.K. Whittier Foundation, the VPICU has become the central resource for pediatric critical care quality, research and education in the United States. Information derived from the VPICU data resources continues to improve the quality of care at Children’s Hospital Los Angeles (CHLA) and throughout the nation—letting the actual evidence from the care of thousands of critically ill children inform the care of the next child.

The VPICU mission

The VPICU mission is to create a common information space for the providers of critical care. This vision preceded Facebook, the social web, Listservs and big data and pioneered the application of machine learning and artificial intelligence to this task. We have created this common information space through web sites, telemedicine, internet-based communication, collaborative research and critical-care quality improvement.

Our successes have decreased the time to discovery and helped improve the quality of care for critically ill children by identifying and sharing best practices and benchmarking excellence. The VPICU has grown to national prominence, expanding the information space using crowd sourcing, collaboration, knowledge dissemination, dynamic content, cloud computing, rich user experiences and, most critically, user-generated content and design. Applications not even imagined when the VPICU was founded are reaching the bedside in the nation’s PICUs.

The VPICU is the pre-eminent communication highway in pediatric critical care and has supported research and quality improvement. It has contributed to saving the lives of tens of thousands of children. However, there is there is much more that we need do to bring the full benefits of the VPICU to the bedside of critically ill children. In the modern information landscape, Big Data is changing health care practice. The VPICU was founded on the concept of big data and has been exploiting big data for over five years with the help of the L.K. Whittier Foundation. The VPICU has created a vast data lake of critical care data from across the United States detailing how critical illness happens to children. The data we have collected is so massive and nearly beyond human comprehension, that it now needs to be managed and made relevant and available in the right place at the right time.

 

A Common Information Space


What is a Common Information Space?

Fifteen years ago, the common information space seemed a strange concept. The space, as imagined then, represented a combination of communication technologies that would bring pediatric critical care together as one Virtual ICU. Since then, this virtual space has been used to care for children geographically distant from academic ICUs, educate caregivers, conduct research and support quality improvement.

Today, the common information space is readily understood as a space where knowledge is shared and is bounded only by the internet and defined by myriad communication technologies developed by the VPICU.

pedsccm screenshot

PEDSCCM

A collaborative, independent, information resource for the PICU community.

www.pedsccm.org

piculist screenshot

PICULIST

Serving caregivers for over two decades, the PICULIST is now its own Slack channel.

piculist.vpicu.net

...

PICUPEDIA

A crowdsourced knowledge base for pediatric critical care medicine.

picupedia.net

 

LEADING THE WAY: Artifical Intelligence in the PICU


 

Nowhere is rapid decision making more urgent, or more complex, than in managing critically ill children. Important to that decision making is the ability to identify similar patients. Patient similarity has long been a challenging thorn to many researchers because of its subjective nature.

The Laura P. and Leland K. Whittier Virtual Pediatric Intensive Care Unit Data Science team works closely with physicians at Children’s Hospital Los Angeles to develop a framework to address this challenge.

The clinical status of an individual patient can be de ned by multiple contexts. To understand a patient’s clinical status, the VPICU Patient Similarity Framework combines these contexts as de ned by corresponding modules. A module generates a representation for each individual patient that enables the computation of a mathematical distance between any two patients within a given clinical context (e.g. mortality, diagnosis, volatility, etc). The summation of the distances from various contexts of interest enables the retrieval of similar patients.

Developing a Dynamic Mortality Module

Severity of illness (SOI) scores have been developed since the early 1980s to aid clinicians assess their critically ill patients. Examples of clinically-used systems are the Pediatric Index of Mortality (PIM) and Pediatric Risk of Mortality (PRISM). Using Recurrent Neural Networks (RNN), the VPICU developed a dynamic mortality model that sequentially ingests and integrates newly available measurements to provide a time-evolving SOI.

The performance of this model and other SOI systems are measured and compared objectively using the accuracy of their in-ICU mortality predictions. As Figure 1 shows, the VPICU’s RNN-based model outperforms PIM, PRISM and two other machine learning models.

Responsive image

Developing a Pediatric Early Warning Module

Patients in hospital wards can undergo critical deterioration which necessitates more aggressive interventions including transfer into an ICU. Leveraging its deep learning expertise, the VPICU developed a new early warning tool for assessing children’s severity of illness (SOI) and detecting their critical decompensation on the general hospital floor. The VPICU’s tool provides significant improvement over a clinically used system, Pediatric Early Warning Score (PEWS), as shown in Figure 2.

Responsive image

Predicting Individual Physiologically Acceptable State Space for Discharge from a Pediatric Intensive Care Unit

Critically ill patients undergo high-frequency monitoring and are treated to achieve and maintain normal homeostasis, often defined by the clinical team's experience and expertise. This includes restoring a patient's vitals towards a medically accepted range of values. In practice, however, these conceptual ‘target’ or ‘normal’ values are often implied but not explicitly defined.

Using historical data from CHLA’s PICU, the VPICU Data Science team described the physiologic state, defined by heart rate and blood pressure that characterizes when a child is well enough, in clinical judgement, to be discharged from the PICU. This physiologically acceptable state space (PASS) for ICU discharge was demonstrated to differ significantly from vital ranges found in the medical literature (see Figure 4). The VPICU’s age-dependent regression model for individual PASS vitals provided more appropriate values. Further, the team applied deep learning methods to develop a model that takes a wider perspective of the patient (beyond age) to predict each child’s PASS vitals more accurately that the published age normals and the age-dependent regression model (see Table 1).

Figure 4. Individual PASS heart rates, as function of age, of children successfully discharged from the PICU (blue circles) are tend to be significantly higher than published age-normals (black dashed and solid lines). A polynomial regression model that depends only on age (blue solid curve) provides more accurate reference values than the published norms.
Responsive image

 

Table 1. Mean errors of different models for predicting the physiologically acceptable state space (PASS) for individual children. The VPICU’s RNN-based model incurred the smallest errors, suggesting that this model can provide more personalized target values.

 

Table 1.
Heart Rate (bpm) Systolic BP (mmHg) Diastolic BP (mmHg)
Age-Normal 21 11 10
Regression 15 10 0
RNN 13 8 7

 

PICU Data Collaborative


Background

The adoption of electronic medical record (EMR) systems has enabled researchers to apply a wide range of emergent machine and deep learning methodologies to the collected data with the goal of discovering and applying knowledge for better diagnosis, prognosis and patient treatment. At the recent AIMED conference in Laguna Nigel CA it was abundantly clear, as it has been to many investigators in health care big data research for years, that one of the major impediments to achieving the ‘big data’ promise of improved care for children, is the lack of sufficiently large datasets. This is particularly true in pediatric critical care. Although we capture and record large amounts of clinical data it is not widely available for researchers. This problem stems from: the lack of suitably large amounts of high quality data; and from the siloed nature of datasets used by various groups and studies, which make it difficult to make “apples-to-apples” comparisons. Furthermore the nature of pediatric critical care, wherein one unit may see a very limited number of certain diagnoses, makes this collaboration even more essential.

To address the problem, we propose a Pediatric Data Collaborative where:
  • Members contribute and share critical care EMR data
  • Data conform to a common schema to facilitate algorithm development, benchmarks and valid standard datasets for development
  • Members gather to share, discuss, and prioritize data aggregation, algorithm development, and clinical deployment

Overview

The PICU Data Collaborative will be comprised of institutions who contribute anonymized pediatric critical care EMR data to a shared data platform which resides in a private cloud-computing environment. Responsive image

Collaborative Technical Overview

Each Collaborative member will be responsible for collecting and aggregating data from its various sources based upon an agreed data schema and element list. The anonymization algorithm will be shared across members to ensure compliance and usability. Standard EMR data (including demographics, diagnoses, bedside and laboratory measurements, medications and procedures for each patient) would be aggregated during the initial phase with the subsequent potential to add additional data including notes, waveform data, imaging data and even genomic data at a later stage as the Collaborative matures. To facilitate new research and development, the data platform will enable automated or semi-automated methods converting the disparate data sets into a format that is easily digestible by data scientists. The platform will also provide artificial intelligence workflows and access to data science workspaces, empowering members to develop the science instead of wrangling with the technicalities of the data.

Common Schema and Benchmarks

Benchmark data sets historically have led to rapid gains in their associated fields. The most noteworthy example of this is the ImageNet database which is freely available and easily accessible to all researchers. Since 2010, annual challenges using this standardized database have led to rapid advances in computer vision capabilities. Well-curated medical datasets that are openly available for benchmarks are scarce, and this dearth makes the tracking of real progress in EMR algorithm development nearly impossible. The MIMIC-III database, which includes vital measurements, laboratory results, notes, fluid balance, procedure codes, diagnoses, imaging reports, among other data, is the most comprehensive and only freely accessible dataset of its kind. It is primarily an adult (16 years or above) critical care database with more than 38,000 patients, but it also contains almost 8,000 neonates. This database has enabled significant progress, but its single-institution nature places uncertainty on the transferability of algorithm developments. Discoveries made from the MIMIC database are rarely validated using databases from other institutions.

Our proposed rich collection of curated and standardized data can be used for benchmarking algorithm development geared towards improving pediatric critical care. The Collaborative could define tasks or problems akin to the ImageNet challenges. Regardless of the task or challenge, a common test set would be set aside to compare algorithms and track progress. Access to the data set will be controlled by the collaborative.

Responsive image
The initial dataset will consist of EMR data from PICU admissions restricted to vitals, lab values, vent settings, drugs, and MAR, of up to 175 individual data points mostly numeric structured data. Waveform, image and unstructured notes will not be included initially. For sites that also participate in VPS linkage to the clinical VPS data is also possible and will provide richer diagnostics, SOI and procedural data. A single data export encompassing as many years as possible retrospectively is initially sought from each collaborative participating member. It is hoped that this endeavor will provide over 100,000 PICU admissions in a static database for research purposes and to demonstrate the value of enhancing and enlarging the dataset.

Symposia for Collaborative Members

An important activity of the Collaborative would be an annual symposium, where researchers and clinicians gather to:

  • Discuss new exciting problems
  • Present results of new approaches on benchmarks
  • hare success stories of implementation
  • Talk about obstacles and how they can be overcome
  • Provide community of researchers working on common problems with common goals
  •  

    PUBLICATIONS


    Deep Learning Recommendation of Treatment from Electronic Data

    Melissa Aczon, Ph.D., David Ledbetter, Long Van Ho, Alec Gunny, Randall Wetzel, M.D. Children’s Hospital Los Angeles, Los Angeles, CA

    knowledge.amia.org

    Estimating Data Requirements to Detect Pediatric Critical Decompensation

    Melissa Aczon, Ph.D.1, David Ledbetter, B.S.1, Sareen Shah, M.D.1 1Children’s Hospital Los Angeles, Los Angeles, CA

    www.amia.org

    Dynamic Mortality Risk Predictions in Pediatric Critical Care Using RNN

    Melissa Aczon, Ph.D., David Ledbetter, Long Van Ho, Alec Gunny, Randall Wetzel, M.D. Children’s Hospital Los Angeles, Los Angeles, CA

    arxiv.org

    Early Prediction Of Patient Deterioration Using Machine Learning Techniques With Time Series Data

    Shah, Sareen; Ledbetter, David; Aczon, Melissa; Flynn, Alysia; Rubin, Sarah

    journals.lww.com

    Predicting Individual Physiologically Acceptable States for Discharge from a Pediatric Intensive Care Unit

    Cameron Carlin, Long Van Ho, David Ledbetter, Melissa Aczon, Randall Wetzel

    arxiv.org

    TEAM


    Responsive image

    Randall Wetzel, MBBS, MRCP, LRCS, MSB, FAAP, FCCM

    Dr. Randall Wetzel is the Director of The Laura P. and Leland K. Whittier VPICU at Children’s Hospital Los Angeles. For Randall C. Wetzel, the quest focuses on artificial intelligence (AI), specifically, informatics engineering novel database systems to support medical decisionmaking. It’s a passion that has inspired the energies of the chief of the Department of Anesthesiology Critical Care Medicine at Children’s Hospital for years. In 1998, Dr. Wetzel pioneered the Laura P. and Leland K. Whittier VPICU, and founded VPS, LLC, which streams the world’s largest database of pediatric critical care patients - letting the actual evidence of thousands and thousands of patients inform the care for the next one.

     

    Dr. Melissa Aczon is a Data Scientist at Children’s Hospital Los Angeles (CHLA), where she is exploiting years of electronic health records collected from the CHLA Pediatric Intensive Care Unit with deep learning methods to create doctor decision aids. She was a Principal Scientist at Areté Associates where she led a team of scientists to develop algorithms that both improve the detection capability and reduce false alarms of a very complex sensor system. Melissa leverages her deep understanding of mathematics to solve signal processing, detection, classification and estimation problems from a wide array of applications. She has worked with data coming from many different types of sensors including radar, optical and acoustic systems. Melissa holds a Bachelor’s Degree in Mathematics from Harvey Mudd College and a Ph.D. in Scientific Computing and Computational Mathematics from Stanford University.
    Brett Bailey is a web developer with over five years of experience designing single-page, mobile-first applications. His focus is on using the latest web technologies to enable rich experiences that, previously, would not have been possible. He has history working in finance, ecommerce, and payment platforms, and Brett believes the healthcare industry is an ideal next space to apply his talents. He has spent the last two years at Children’s Hospital Los Angeles creating web applications in order to improve patient care.
    Cameron Carlin is a Data Scientist at Children’s Hospital Los Angeles (CHLA) where he implements machine learning and deep learning techniques on electronic health records (EHR) from the past 10 years of Pediatric ICU episodes. His work at VPICU focuses on developing algorithms targeting patient-specific care to improve understanding of the temporal nature of patient well-being. He is well-versed in exploratory data analysis, statistical modeling, probability theory and algorithm development. Cameron holds Bachelor’s Degrees in Statistics and Psychology from the University of California, Santa Barbara and a Master’s Degree in Analytics from the University of San Francisco.
    Dr. Flynn is a software development engineer with a PhD in Computer Science and Informatics from Cardiff University. A member of the data systems team, she designs and develops data integration and manipulation processes to support the team’s applications.
    Long Van Ho is a Data Scientist who has had extensive experience with applications of advanced machine learning techniques in health and defense. He has been involved in developing a recommender algorithm and system for Heritage Provider Network to improve overall health care for their patients and decrease potential costs and risks. He has also had experienced with predicting dengue fever occurrences, placing top 6 in the NOAA’s Dengue Fever Forecasting Competition in 2015. Long’s background in physics and research experience at UCLA and Stanford has provided a strong scientific background in his career as a data scientist. His interests and goal is to bridge the potential of machine learning with practical applications to health. Presently, Long is working at Children’s Hospital Los Angeles and the Pediatric ICU to develop algorithms and applications for real-time clinical support for doctors in the ICU.
    Eugene Laksana works as a Data Scientist at Children’s Hospital Los Angeles (CHLA) where he leverages deep learning techniques on 10 years of electronic health records (EHR) to develop algorithms for state-of-the-art clinical decision support. He is experienced with handling multimodal data and has worked on projects ranging from investigating temporal decay in EHR data value at CHLA to detecting suicide ideation using facial tracking software and machine learning at Carnegie Mellon University. Eugene holds. a B.Sc in Computer Science from the University of Southern California. He really likes candy.
    David Ledbetter has an extensive and deep understanding of decision theory. He has experience implementing various decision engines, including convolutional neural networks, random forests, extra trees, and linear discrimination analysis. His particular area of focus is in performance estimation, where he has demonstrated a tremendous ability to accurately predict performance on new data in nonstationary, real-world scenarios. David has worked on a number of real-world detection projects, including detecting circulating tumor cells in blood, automatic target recognition utilizing CNNs from satellite imagery, make/model car classification for the Los Angeles Police Department using CNNs, and acoustic right whale call detection from underwater sonobuoys. Recently, David has been developing a CNN to generate personalized treatment recommendations to optimize patient outcomes using unstructured electronic medical records from 10 years of data collected from the Children’s Hospital Los Angeles Pediatric Intensive Care Unit.
    Mohit Mehra is a principal data architect with over 15 years experience in Software Engineering and Solution Architecture. He has led several projects in the Healthcare space from inception to delivery. Mohit has worked extensively on real-time applications ranging from Patient Monitoring and Cardiology devices to creating Health Information Exchanges. His particular area of focus is in developing a distributed data platform to support machine learning based research and real-time applications. Recently, he has been involved in creating a Hadoop based secure cloud platform capable of ingesting, transforming, indexing and storing high-frequency data acquired from various hospital source systems such as ICU monitors, Laboratory systems, EMR and other ancillary SQL databases. Mohit holds a masters in computer science from Texas A&M University and a bachelors in Electronics Engineering from India.
    Mike Reilly currently manages the cloud infrastructure and software supporting the data team of The Laura P. and Leland K. Whittier Virtual PICU at CHLA. Mike has an extensive background planning, deploying, and managing public and private cloud environments covering servers, storage, and application virtualization. His past work includes managing projects and creating solutions across a diverse background of industries including banking, entertainment, healthcare, and logistics. Mike holds a Bachelor’s degree in Business Administration with specialties in both Business Management and Information Systems.
    Paul has been with the team since 2002. In that span he was the technical lead on two telemedicine grants and developed several web-based applications used in research studies. Currently, Paul serves as the VPICU’s Senior Program Manager. Paul’s main focus is managing the completion of all key objectives ensuring the success of The Laura P. and Leland K. Whittier VPICU.

    Need more information?


    If you have any questions regarding The Laura P. and Leland K. Whittier Virtual PICU, please send us an email.