Skip to content

In MSQT, increased the upper bound on the expected number of metric samples when the test runs long #276

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bieryAtFnal merged 2 commits into from
Dec 15, 2025
Merged

In MSQT, increased the upper bound on the expected number of metric samples when the test runs long #276

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1d7d87f
In minimal_system_quick_test.py, added functionality to increase the …
Dec 10, 2025
14cd266
Updated the calculation of expected metric samples in minimal_system_…
Dec 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 22 additions & 1 deletion integtest/minimal_system_quick_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,14 +157,35 @@ def test_data_files(run_nanorc):
def test_metric_files(run_nanorc):
print("") # Clear potential dot from pytest

# 10-Dec-2025, KAB: we have noticed that sometimes drunc transitions (or other parts of
# a run control session) take a little longer than expected. This can cause extra metric
# samples to be created. This section of code takes that into account by increasing
# the max allowed sample count by the amount of extra time taken, divided by 10
# (metric samples are produced every 10 seconds, by default).
# I've tried to make this code backward compatible by handling cases in which the
# daq_session_overall_time is not available (e.g. the try/catch).
max_metric_sample_count = 5 # under normal conditions, the sample count is 3, so 5 allows for some variation
try:
#print(f"DAQ session overall time: {run_nanorc.daq_session_overall_time} seconds")
expected_daq_session_time = 40 # this was determined by looking at the overall time from a normal run
if run_nanorc.daq_session_overall_time is not None:
extra_time_taken = run_nanorc.daq_session_overall_time - expected_daq_session_time
if extra_time_taken > 10:

@eflumerf eflumerf Dec 11, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should a warning message be printed here? Should we measure expected_daq_session_time across several different run_duration values, and see if it scales in an expected way? (And if so, should expected_daq_session_time be expressed as a function of run_duration?)

extra_sample_count_allowance = int(extra_time_taken / 10)
max_metric_sample_count += extra_sample_count_allowance
except AttributeError:
pass
#print(f"max_metric_sample_count: {max_metric_sample_count}")

session_name = run_nanorc.session_name if run_nanorc.session_name else run_nanorc.session
metric_data = opmon_metric_checks.collate_opmon_data_from_files(run_nanorc.opmon_files)

metric_key_list = [session_name, "df-01", "df-01-trb", "dfmodules.TRBInfo", "generated_trigger_records"]
all_ok = True
# a 20-second run will likely result in 3 metric samples (at 10-second intervals), so a range
# of 1..5 should always succeed
all_ok &= opmon_metric_checks.check_metric_sample_count(metric_data, metric_key_list, min_count=1, max_count=5)
all_ok &= opmon_metric_checks.check_metric_sample_count(metric_data, metric_key_list, min_count=1,
max_count=max_metric_sample_count)
# the number of triggers expected in this test is ~20, so a test that checks for the reported
# number of generated trigger records between 17 and 23 shoudl always succeed
all_ok &= opmon_metric_checks.check_metric_value_sum(metric_data, metric_key_list, min_value_sum=17, max_value_sum=23)
Expand Down