The counting processes are features that increment by 1 every time a brand new occasion arrives. Clearly, there are fewer occasions occurring within the remedy than within the management. If these have been login occasions, this is able to counsel that the brand new code accommodates a bug that stops some customers from having the ability to log in efficiently.
This can be a frequent scenario when coping with occasion timestamps. To present one other instance, if occasions corresponded to errors or crashes, we wish to know if these are accruing quicker within the remedy than within the management. Furthermore, we need to reply that query as rapidly as potential to stop any additional disruption to the service. This necessitates sequential testing methods which have been launched partly 1.
Time-Inhomogeneous Poisson Course of
Our knowledge for every remedy group is a realization of a one-dimensional level course of, that’s, a sequence of timestamps. As the speed at which the occasions arrive is time-varying (in each remedy and management), we mannequin the purpose course of as a time-inhomogeneous Poisson level course of. This level course of is outlined by an depth perform λ: ℝ → [0, ∞). The number of events in the interval [0,t), denoted N(t), has the following Poisson distribution
N(t) ~ Poisson(Λ(t)), where Λ(t) = ∫₀ᵗ λ(s) ds.
We seek to test the null hypothesis H₀: λᴬ(t) = λᴮ(t) for all t i.e. the intensity functions for control (A) and treatment (B) are the same. This can be done semiparametrically without making any assumptions about the intensity functions λᴬ and λᴮ. Moreover, the novelty of the research is that this can be done sequentially, as described in section 4 of our paper. Conveniently, the only data required to test this hypothesis at time t is Nᴬ(t) and Nᴮ(t), the total number of events observed so far in control and treatment. In other words, all you need to test the null hypothesis is two integers, which can easily be updated as new events arrive. Here is an example from a simulated A/A test, in which we know by design that the intensity function is the same for the control (A) and the treatment (B), albeit nonstationary.
