Theodolite Executions look similar to the following example.
apiVersion: theodolite.rocks/v1beta1 kind: execution metadata: name: theodolite-example-execution spec: benchmark: "example-benchmark" load: loadType: "NumSensors" loadValues: [25000, 50000] resources: resourceType: "Instances" resourceValues: [1, 2] slos: - name: "lag trend" properties: threshold: 2000 execution: metric: "demand" strategy: name: "RestrictionSearch" restrictions: - "LowerBound" searchStrategy: "LinearSearch" duration: 300 # in seconds repetitions: 1 loadGenerationDelay: 30 # in seconds configOverrides: - patcher: type: "SchedulerNamePatcher" resource: "uc1-kstreams-deployment.yaml" value: "random-scheduler"
Similar to Kubernetes Jobs, Theodolite Executions are not automatically deleted after they are finished. Therefore, it is recommended to choose a meaningful name for your Execution. Additionally, you can add labels to your Executions as with any other Kubernetes resource.
An Execution always refers to a Benchmark. For the Execution to run, the Benchmark must be registered with Kubernetes and it must be in state Ready. If this is not the case, the Execution will remain in state Pending.
As a Benchmark may define multiple supported load and resource types, an Execution has to pick exactly one of each by its name. Additionally, it defines the set of load values and resource values the benchmark should be executed with.
Both these values are represented as integers, which are interpreted in a Benchmark-specific way to configure the SUT and load generator.
Similarly, an Execution must select a subset of the SLOs defined in the Benchmark. Additionally, these SLOs can be configured by their
According to Theodolite’s measurement method, isolated SLO experiments are performed for different combinations of load intensity and resource amounts. The experimental setup can be configured by:
- A scalability metric (
metric). Supported values are
demandbeing the default.
- A search strategy (
strategy), which determines which load and resource combinations should be tested. Supported values are
BinarySearch. Additionally, a
restrictionscan be set to
durationper SLO experiment in seconds.
- The number of repetitions (
repetitions) for each SLO experiment.
loadGenerationDelay, specifying the time in seconds before the load generation starts.
In cases where only small modifications of a system under test should be benchmarked, it is not necessary to create a new benchmark. Instead, also Executions allow to do small reconfigurations, such as switching on or off a specific Pod scheduler.
This is done by defining
configOverrides in the Execution. Each override consists of a patcher, defining which Kubernetes resource should be patched in which way, and a value the patcher is applied with.