Introduction
Most clinical outcome assessment in drug and device development relies on a single primary informant: either the participant (patient-reported), a clinician (observer-rated), or an objective measurement instrument. The data management infrastructure that supports those studies reflects that assumption. Forms are assigned to one role. Data flows through one collection pathway. The analysis merges a relatively uniform data type.
Communication research in autism spectrum disorder does not operate under that assumption. Measuring communication outcomes in nonspeaking and minimally verbal autistic individuals requires simultaneous data from at least three distinct informant streams: a clinician or speech-language pathologist observing behavior in structured sessions, a caregiver reporting functional outcomes across home and community contexts, and a device or application logging interactions directly. These three streams are not measuring the same thing from different perspectives. They are measuring related but structurally distinct constructs, in different contexts, at different time granularities, with different systematic biases.
The multi-informant problem in ASD communication research is not a logistics challenge. It is a measurement validity challenge with direct consequences for how endpoints are defined, how data systems are architected, and how results are interpreted.
Why Three Informant Streams Are Necessary
The case for multiple informants is grounded in the nature of the construct being measured. Communication is not a trait that exists uniformly across contexts. An autistic child who communicates effectively with a familiar caregiver in a structured home environment may demonstrate markedly different communication patterns with an unfamiliar clinician in a clinical setting. Neither observation is wrong. They are measuring communication in different contexts, and both are relevant to the research question.
Caregiver-reported outcomes capture the ecological validity dimension: how the participant communicates in the environments that matter most to daily life, across a range of partners, over time. Caregivers have longitudinal familiarity with the participant's communication patterns, idiosyncratic expressions, and contextual variation. They are also subject to well-documented biases: expectation effects, response shift over the course of an intervention, and the difficulty of distinguishing change in the participant from change in the caregiver's interpretation of the participant's behavior.
Clinician observation in structured sessions provides standardized conditions that allow for comparison across participants and phases. A trained SLP administering a validated rating scale in a defined session protocol produces data that is more comparable across sites and raters than caregiver report. It is also less ecologically valid: the session context is artificial, the communication partner is often unfamiliar, and the assessment period is brief. What a clinician observes in a 20-minute session may not represent typical communication across the participant's day.
Device-instrumented data occupies a third position entirely. When a participant's AAC device or communication application logs every interaction, the resulting dataset is continuous, objective, and independent of any human rater's judgment. It captures session-level behavior with precision that neither caregiver report nor clinician observation can match. It is also blind to context: a logged utterance carries no information about whether it was spontaneous or prompted, whether it was communicatively effective, or whether it reflected a genuine communicative intent or a repetitive interaction pattern.
None of these streams is sufficient alone. Together, they produce a measurement model that is more valid than any single informant could achieve. That validity comes at a cost: the data streams are structurally heterogeneous, collected on different schedules, and subject to different reliability properties. Managing them as a unified research dataset requires infrastructure that was not designed for standard eCOA administration.
The Structural Heterogeneity Problem
The three informant streams differ not only in who provides the data but in how it is collected, at what frequency, and in what format.
Device-instrumented data is continuous and session-level. Every button press, every suggestion accepted or rejected, every completed utterance is logged in real time. The resulting dataset is high-dimensional: hundreds or thousands of events per participant per session, structured as a timestamped interaction log. This data does not look like a clinical outcome form. It looks like application telemetry.
Clinician-rated data is collected at discrete session timepoints, typically every session or at phase transitions, by a trained rater completing a structured instrument. It is ordinal, low-dimensional, and dependent on rater training and inter-rater reliability protocols to be interpretable. The same 20-minute session that generates hundreds of device-log events produces a handful of Likert-scale ratings.
Caregiver-reported data is collected at fixed study timepoints: baseline, post-intervention, and maintenance. It is retrospective, summarizing the participant's communication behavior over weeks or months rather than capturing a specific session. The time granularity is coarser than either of the other streams, and the instrument is designed to capture functional participation rather than moment-to-moment behavior.
The challenge for a clinical data platform is that these three data types cannot be managed through the same collection architecture. Device-instrumented data requires a structured import pathway: the application logs are exported in a defined schema, validated on import, and merged into the study dataset alongside the clinical outcome data. Clinician-rated data requires a session-linked eCOA workflow that associates each rating with the correct session, phase, and participant. Caregiver-reported data requires a separate administration schedule, a different instrument, and a different role-based access configuration.
In a standard EDC system, these requirements are difficult to satisfy simultaneously. Session-linked eCOA administration is not the same as visit-linked eCOA administration. Structured app log import requires a custom integration pathway that most EDC platforms were not designed to handle. Role-based access must be configured separately for each informant type, with appropriate blinding controls where the study design requires them.
The Convergent Validity Challenge
Even when the three data streams are successfully collected and merged, the analysis faces a convergent validity problem: the streams often do not agree, and disagreement is not necessarily a sign that something went wrong.
A participant who shows a measurable increase in device-logged emotional vocabulary frequency may show little change on a clinician-rated emotional expression scale, because the structured session context does not elicit the same communication patterns as the participant's natural environment. A caregiver may report substantially improved communication participation at home while session-based measures show modest gains, because caregiver familiarity allows them to detect subtle communicative cues that a clinician rater misses.
These discrepancies carry methodological information. Low correlation between device-logged and clinician-rated endpoints may reflect genuine context-dependency in the intervention's effects rather than measurement error. High caregiver report scores with modest session-based scores may indicate that the most meaningful outcomes of an AAC intervention are visible in everyday contexts but not in structured clinical sessions.
Interpreting these patterns requires a pre-specified analytic framework that treats convergent validity as a research question rather than assuming it. Studies that average across informants without accounting for structural differences between the streams are not producing a more reliable estimate. They are obscuring the context-specificity of the effect.
This is a design and analysis challenge that the research community has not fully resolved. What it requires from data infrastructure is that each stream is captured with sufficient fidelity and metadata to support the convergent validity analysis: session identifiers that link device logs to clinician ratings, timing information that allows caregiver reports to be mapped to the correct study phase, and a unified data model that preserves the structure of each stream rather than flattening it into a single outcome table.
What the Infrastructure Needs to Do
The data infrastructure requirements for a multi-informant ASD communication study are materially different from the requirements for a standard eCOA-based Phase 2 trial.
The platform must support concurrent administration of instruments across at least two distinct roles, clinician and caregiver, with different administration schedules, different instruments, and different access permissions. A clinician completing a session-linked rating needs to see a workflow that associates the rating with the correct session and phase. A caregiver completing a periodic outcome measure needs a separate workflow calibrated to the study's fixed assessment timepoints.
The platform must support structured import of device-instrumented data. This is not a standard eCOA function. It requires a defined schema for the application log, a validation layer that checks incoming data against that schema on import, and a storage architecture that preserves the raw log while making derived summary metrics available for analysis. The import must be linked to session identifiers so that device-logged events can be associated with the correct clinician observation from the same session.
The platform must support the blinding requirements of the study design. In a multiple-baseline single-case experimental design, where participants enter the intervention phase at staggered timepoints, the phase assignment for each participant must be managed by the platform rather than communicated through the site team. If clinician raters are not blinded to phase, that limitation must be documented in the study protocol and reflected in the platform's audit trail.
And the platform must produce a unified data export that preserves the structural relationships between streams: which device log sessions correspond to which clinician ratings, which caregiver assessments cover which study phases, and which participants were in which phase at each timepoint. Without that structure, the convergent validity analysis cannot be conducted.
An event-driven clinical data platform, one that captures every protocol-relevant event in real time and links it to the session, phase, and participant context it belongs to, provides the underlying architecture for all of these requirements. The three informant streams become three data layers within a single operational model, each collected on its own schedule and through its own pathway, but linked to a common temporal and participant structure that makes the convergent validity analysis tractable.
Conclusion
The multi-informant measurement problem in ASD communication research is one of the most technically demanding data management challenges in behavioral intervention studies. It requires simultaneous management of continuous device-logged data, session-linked clinician ratings, and periodic caregiver reports, each collected on a different schedule, by a different informant, with a different reliability profile and a different relationship to the underlying construct.
Standard clinical data infrastructure was not designed for this. The assumption of a single primary informant stream, administered on a visit-linked schedule, is embedded in the architecture of most eCOA platforms and EDC systems. Multi-informant research in ASD requires a platform built around a different assumption: that the study record is the product of multiple concurrent data streams, each with its own collection logic, and that the analysis depends on preserving the structural relationships between them.
As behavioral intervention research in ASD advances, and as the evidence standards for communication outcomes become more rigorous, the data infrastructure supporting these studies will need to keep pace with the methodological demands the research questions create.
