Lecture
In statistics, censoring is a condition in which the value of a measurement or observation is only partially known.
For example, suppose a study is being conducted on the effect of a drug on mortality. In such a study it may be known that the age of a deceased person is at least 75 years (but could be more). Such a situation can arise if the person dropped out of the study at age 75, or if the person is currently alive and is 75 years old.
Censoring also occurs when a value falls outside the range of a measuring instrument. For example, a floor scale may measure only up to 140 kg, after which it rolls over past 0 and continues counting. If a person weighing 160 kg is weighed on this scale, the observer will only know that their weight equals 20 mod 140 kg (besides 160 kg, they could weigh 20 kg, 300 kg, 440 kg, and so on).
The problem of censored data, in which the observed value of some variable is only partially known, is related to the problem of missing data, in which the observed value of some variable is unknown.
Censoring should not be confused with the related idea of truncation. With censoring, observations either allow the exact applicable value to be known, or determine that the value lies within an interval. With truncation, observations never fall outside a given range: population values outside the range are never observed, or if they are observed, they are never recorded. Note that in statistics, truncation is not the same thing as rounding.
Censoring is a form of the missing data problem in which the time to an event is not tracked, for reasons such as the study ending before all enrolled participants have shown the event of interest, or a participant leaving the study before the event occurred. Censoring is commonly encountered in survival analysis.
If only a lower bound l is known for the true event time T , with T > l , this is called right censoring . Right censoring will occur, for example, for those subjects whose birth date is known but who are still alive at the time of loss to follow-up or at the end of the study. We commonly encounter data that are right-censored.
If the event of interest has already occurred before the subject was enrolled in the study, but it is unknown when it occurred, the data are said to be subject to left censoring . [ 24 ] When it can only be said that the event occurred between two observations or examinations, this is interval censoring .
Interval censoring can occur when observing a value requires additional observations or checks. Left and right censoring are special cases of interval censoring, where the start of the interval corresponds to zero and the end to infinity, respectively.
Estimation methods for using left-censored data vary, and not all estimation methods may be applicable or most reliable for all data sets. [ 1 ]
A common misconception about time-interval data is that intervals whose starting time is unknown are classified as left-censored . In such cases we have a lower bound on the time interval ; thus, the data are right-censored (even though the missing starting point lies to the left of the known interval, when viewed as a timeline!).
Special methods can be used to handle censored data. Tests with a definite failure time are coded as actual failures; censored data are coded taking into account the type of censoring and the known interval or bound. Special programs (often reliability-oriented) can perform maximum-likelihood estimation for summary statistics, confidence intervals, and so on.
One of the earliest attempts to analyze a statistical problem involving censored data was the analysis of smallpox morbidity and mortality data conducted by Daniel Bernoulli in 1766 to demonstrate the effectiveness of vaccination. Early work on using the Kaplan-Meier estimator to estimate censored costs was written by Quesenberry et al. (1989) , however Lin et al. considered this approach invalid , unless all patients accumulated costs with a common deterministic rate function over time, and they proposed an alternative estimation method known as the Lin estimator.
Example of five repeated tests resulting in four failures and one suspension, resulting in censoring.
Reliability testing often consists of conducting an item test (under specified conditions) to determine the time required for a failure to occur.
Analysis of repeated-test data includes both the time-to-failure for defective items and the time-to-suspension for those that did not fail.
An earlier censored regression model, the Tobit model, was proposed by James Tobin in 1958. [
Likelihood is the probability or probability density of the observed event, considered as a function of the parameters of the assumed model. To include censored data points in the likelihood, censored data points are represented by the probability of the censored data points as a function of the model parameters, i.e. by the CDF(s) function, rather than by the density or probability mass.
The most general case of censoring is interval censoring: 
, where 
— is the cumulative distribution function, and two special cases:
For continuous probability distributions: 
Suppose we are interested in the survival times,Т1,Т2,...,Тн
, but we do not observeТя
for allя
. Instead we observe
, with 
and 
if is actually observed, and
, with 
and 0
if all we know is that is longer than 
.
When 
is called the censoring time .
If the censoring times are known constants, then the likelihood is
where 
= the probability density function evaluated at 
,
and 
= the probability that is greater than , called the survival function .
This can be simplified by defining the hazard function, the instantaneous force of mortality, as
so
.
Then
.
For the exponential distribution this becomes even simpler, since the hazard rate,λ
, is a constant, and 
. Then:
,
whereк 
.
From this we easily computeλ^
, the maximum likelihood estimate (MLE)λ, as follows:
.
Then
.
We set this to 0 and solve forλto obtain:
.
Equivalently, the mean time to failure is:
.
This differs from the standard MLE for the exponential distribution in that censored observations are counted only in the numerator.
Comments