Criterion.Measurement

criterion-measurement-0.2.3.0: Criterion measurement functionality and associated types

Copyright | (c) 2009-2014 Bryan O'Sullivan
License | BSD-style
Maintainer | bos@serpentine.com
Stability | experimental
Portability | GHC
Safe Haskell | Trustworthy
Language | Haskell2010

Criterion.Measurement

Description

Benchmark measurement code.

Synopsis

- initializeTime :: IO ()
- getTime :: IO Double
- getCPUTime :: IO Double
- getCycles :: IO Word64
- getGCStatistics :: IO (Maybe GCStatistics)
- data GCStatistics = GCStatistics {
  - gcStatsBytesAllocated :: !Int64
  - gcStatsNumGcs :: !Int64
  - gcStatsMaxBytesUsed :: !Int64
  - gcStatsNumByteUsageSamples :: !Int64
  - gcStatsCumulativeBytesUsed :: !Int64
  - gcStatsBytesCopied :: !Int64
  - gcStatsCurrentBytesUsed :: !Int64
  - gcStatsCurrentBytesSlop :: !Int64
  - gcStatsMaxBytesSlop :: !Int64
  - gcStatsPeakMegabytesAllocated :: !Int64
  - gcStatsMutatorCpuSeconds :: !Double
  - gcStatsMutatorWallSeconds :: !Double
  - gcStatsGcCpuSeconds :: !Double
  - gcStatsGcWallSeconds :: !Double
  - gcStatsCpuSeconds :: !Double
  - gcStatsWallSeconds :: !Double
  }
- secs :: Double -> String
- measure :: Benchmarkable -> Int64 -> IO (Measured, Double)
- runBenchmark :: Benchmarkable -> Double -> IO (Vector Measured, Double)
- runBenchmarkable :: Benchmarkable -> Int64 -> (a -> a -> a) -> (Int64 -> IO () -> IO a) -> IO a
- runBenchmarkable_ :: Benchmarkable -> Int64 -> IO ()
- measured :: Measured
- applyGCStatistics :: Maybe GCStatistics -> Maybe GCStatistics -> Maybe GCStatistics -> Measured -> Measured
- threshold :: Double

# Documentation

initializeTime :: IO () #

Set up time measurement.

`criterion` measures time using OS-specific APIs whenever possible for efficiency. On certain operating systems, such as macOS and Windows, one must explicitly initialize a timer (which `initializeTime` accomplishes) before one can actually measure the current time (which `getTime` accomplishes).

It is imperative that you call `initializeTime` before calling `getTime`. (See this bug report for an example of what can happen if you do not do so.) All of the `IO`-returning functions in Criterion.Main make sure that this is done, but other functions (such as those in Criterion.Measurement) do not guarantee this unless otherwise stated.

getTime :: IO Double #

Return the current wallclock time, in seconds since some arbitrary time.

You must call `initializeTime` once before calling this function! Refer to the documentation for `initializeTime` for more details.

getCPUTime :: IO Double #

Return the amount of elapsed CPU time, combining user and kernel (system) time into a single measure.

getCycles :: IO Word64 #

Read the CPU cycle counter.

getGCStatistics :: IO (Maybe GCStatistics) #

Try to get GC statistics, bearing in mind that the GHC runtime will throw an exception if statistics collection was not enabled using "`+RTS -T`".

If you need guaranteed up-to-date stats, call `performGC` first.

data GCStatistics #

Statistics about memory usage and the garbage collector. Apart from `gcStatsCurrentBytesUsed` and `gcStatsCurrentBytesSlop` all are cumulative values since the program started.

`GCStatistics` is cargo-culted from the `GCStats` data type that GHC.Stats used to export. Since `GCStats` was removed in GHC 8.4, `criterion` uses `GCStatistics` to provide a backwards-compatible view of GC statistics.

Constructors

GCStatistics |
Fields gcStatsBytesAllocated :: !Int64 Total number of bytes allocated gcStatsNumGcs :: !Int64 Number of garbage collections performed (any generation, major and minor) gcStatsMaxBytesUsed :: !Int64 Maximum number of live bytes seen so far gcStatsNumByteUsageSamples :: !Int64 Number of byte usage samples taken, or equivalently the number of major GCs performed. gcStatsCumulativeBytesUsed :: !Int64 Sum of all byte usage samples, can be used with `gcStatsNumByteUsageSamples` to calculate averages with arbitrary weighting (if you are sampling this record multiple times). gcStatsBytesCopied :: !Int64 Number of bytes copied during GC gcStatsCurrentBytesUsed :: !Int64 Number of live bytes at the end of the last major GC gcStatsCurrentBytesSlop :: !Int64 Current number of bytes lost to slop gcStatsMaxBytesSlop :: !Int64 Maximum number of bytes lost to slop at any one time so far gcStatsPeakMegabytesAllocated :: !Int64 Maximum number of megabytes allocated gcStatsMutatorCpuSeconds :: !Double CPU time spent running mutator threads. This does not include any profiling overhead or initialization. gcStatsMutatorWallSeconds :: !Double Wall clock time spent running mutator threads. This does not include initialization. gcStatsGcCpuSeconds :: !Double CPU time spent running GC gcStatsGcWallSeconds :: !Double Wall clock time spent running GC gcStatsCpuSeconds :: !Double Total CPU time elapsed since program start gcStatsWallSeconds :: !Double Total wall clock time elapsed since start

#### Instances

Instances details

Data GCStatistics # |
Instance details Defined in Criterion.Measurement Methods gfoldl :: (forall d b. Data d => c (d -> b) -> d -> c b) -> (forall g. g -> c g) -> GCStatistics -> c GCStatistics # gunfold :: (forall b r. Data b => c (b -> r) -> c r) -> (forall r. r -> c r) -> Constr -> c GCStatistics # toConstr :: GCStatistics -> Constr # dataTypeOf :: GCStatistics -> DataType # dataCast1 :: Typeable t => (forall d. Data d => c (t d)) -> Maybe (c GCStatistics) # dataCast2 :: Typeable t => (forall d e. (Data d, Data e) => c (t d e)) -> Maybe (c GCStatistics) # gmapT :: (forall b. Data b => b -> b) -> GCStatistics -> GCStatistics # gmapQl :: (r -> r' -> r) -> r -> (forall d. Data d => d -> r') -> GCStatistics -> r # gmapQr :: forall r r'. (r' -> r -> r) -> r -> (forall d. Data d => d -> r') -> GCStatistics -> r # gmapQ :: (forall d. Data d => d -> u) -> GCStatistics -> [u] # gmapQi :: Int -> (forall d. Data d => d -> u) -> GCStatistics -> u # gmapM :: Monad m => (forall d. Data d => d -> m d) -> GCStatistics -> m GCStatistics # gmapMp :: MonadPlus m => (forall d. Data d => d -> m d) -> GCStatistics -> m GCStatistics # gmapMo :: MonadPlus m => (forall d. Data d => d -> m d) -> GCStatistics -> m GCStatistics #
Generic GCStatistics # |
Instance details Defined in Criterion.Measurement Associated Types
type Rep GCStatistics |
Instance details Defined in Criterion.Measurement type Rep GCStatistics = D1 ('MetaData "GCStatistics" "Criterion.Measurement" "criterion-measurement-0.2.3.0-GICzKqVZCZFbK3A27iKbD" 'False) (C1 ('MetaCons "GCStatistics" 'PrefixI 'True) ((((S1 ('MetaSel ('Just "gcStatsBytesAllocated") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64) :*: S1 ('MetaSel ('Just "gcStatsNumGcs") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64)) :*: (S1 ('MetaSel ('Just "gcStatsMaxBytesUsed") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64) :*: S1 ('MetaSel ('Just "gcStatsNumByteUsageSamples") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64))) :*: ((S1 ('MetaSel ('Just "gcStatsCumulativeBytesUsed") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64) :*: S1 ('MetaSel ('Just "gcStatsBytesCopied") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64)) :*: (S1 ('MetaSel ('Just "gcStatsCurrentBytesUsed") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64) :*: S1 ('MetaSel ('Just "gcStatsCurrentBytesSlop") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64)))) :*: (((S1 ('MetaSel ('Just "gcStatsMaxBytesSlop") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64) :*: S1 ('MetaSel ('Just "gcStatsPeakMegabytesAllocated") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64)) :*: (S1 ('MetaSel ('Just "gcStatsMutatorCpuSeconds") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double) :*: S1 ('MetaSel ('Just "gcStatsMutatorWallSeconds") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double))) :*: ((S1 ('MetaSel ('Just "gcStatsGcCpuSeconds") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double) :*: S1 ('MetaSel ('Just "gcStatsGcWallSeconds") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double)) :*: (S1 ('MetaSel ('Just "gcStatsCpuSeconds") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double) :*: S1 ('MetaSel ('Just "gcStatsWallSeconds") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double))))))
Methods from :: GCStatistics -> Rep GCStatistics x # to :: Rep GCStatistics x -> GCStatistics #
Read GCStatistics # |
Instance details Defined in Criterion.Measurement Methods readsPrec :: Int -> ReadS GCStatistics # readList :: ReadS [GCStatistics] # readPrec :: ReadPrec GCStatistics # readListPrec :: ReadPrec [GCStatistics] #
Show GCStatistics # |
Instance details Defined in Criterion.Measurement Methods showsPrec :: Int -> GCStatistics -> ShowS # show :: GCStatistics -> String # showList :: [GCStatistics] -> ShowS #
Eq GCStatistics # |
Instance details Defined in Criterion.Measurement Methods (==) :: GCStatistics -> GCStatistics -> Bool # (/=) :: GCStatistics -> GCStatistics -> Bool #
type Rep GCStatistics # |
Instance details Defined in Criterion.Measurement type Rep GCStatistics = D1 ('MetaData "GCStatistics" "Criterion.Measurement" "criterion-measurement-0.2.3.0-GICzKqVZCZFbK3A27iKbD" 'False) (C1 ('MetaCons "GCStatistics" 'PrefixI 'True) ((((S1 ('MetaSel ('Just "gcStatsBytesAllocated") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64) :*: S1 ('MetaSel ('Just "gcStatsNumGcs") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64)) :*: (S1 ('MetaSel ('Just "gcStatsMaxBytesUsed") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64) :*: S1 ('MetaSel ('Just "gcStatsNumByteUsageSamples") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64))) :*: ((S1 ('MetaSel ('Just "gcStatsCumulativeBytesUsed") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64) :*: S1 ('MetaSel ('Just "gcStatsBytesCopied") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64)) :*: (S1 ('MetaSel ('Just "gcStatsCurrentBytesUsed") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64) :*: S1 ('MetaSel ('Just "gcStatsCurrentBytesSlop") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64)))) :*: (((S1 ('MetaSel ('Just "gcStatsMaxBytesSlop") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64) :*: S1 ('MetaSel ('Just "gcStatsPeakMegabytesAllocated") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Int64)) :*: (S1 ('MetaSel ('Just "gcStatsMutatorCpuSeconds") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double) :*: S1 ('MetaSel ('Just "gcStatsMutatorWallSeconds") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double))) :*: ((S1 ('MetaSel ('Just "gcStatsGcCpuSeconds") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double) :*: S1 ('MetaSel ('Just "gcStatsGcWallSeconds") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double)) :*: (S1 ('MetaSel ('Just "gcStatsCpuSeconds") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double) :*: S1 ('MetaSel ('Just "gcStatsWallSeconds") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double))))))

secs :: Double -> String #

Convert a number of seconds to a string. The string will consist of four decimal places, followed by a short description of the time units.

measure #

Arguments

:: Benchmarkable | Operation to benchmark.
-> Int64 | Number of iterations.
-> IO (Measured, Double) |

Measure the execution of a benchmark a given number of times.

This function initializes the timer before measuring time (refer to the documentation for `initializeTime` for more details).

runBenchmark #

Arguments

:: Benchmarkable |
-> Double | Lower bound on how long the benchmarking process should take. In practice, this time limit may be exceeded in order to generate enough data to perform meaningful statistical analyses.
-> IO (Vector Measured, Double) |

Run a single benchmark, and return measurements collected while executing it, along with the amount of time the measurement process took.

This function initializes the timer before measuring time (refer to the documentation for `initializeTime` for more details).

runBenchmarkable :: Benchmarkable -> Int64 -> (a -> a -> a) -> (Int64 -> IO () -> IO a) -> IO a #

runBenchmarkable_ :: Benchmarkable -> Int64 -> IO () #

measured :: Measured #

An empty structure.

applyGCStatistics #

Arguments

:: Maybe GCStatistics | Statistics gathered at the end of a run, post-GC.
-> Maybe GCStatistics | Statistics gathered at the end of a run, pre-GC.
-> Maybe GCStatistics | Statistics gathered at the beginning of a run.
-> Measured | Value to "modify".
-> Measured |

Apply the difference between two sets of GC statistics to a measurement.

threshold :: Double #

The amount of time a benchmark must run for in order for us to have some trust in the raw measurement.

We set this threshold so that we can generate enough data to later perform meaningful statistical analyses.

The threshold is 30 milliseconds. One use of `runBenchmark` must accumulate more than 300 milliseconds of total measurements above this threshold before it will finish.