Profiling middleware - #1228
Conversation
aagbsn
left a comment
aagbsn
left a comment
There was a problem hiding this comment.
this looks useful. I am wondering if the ProfileMiddleware should get added via app_middleware only if profiling is enabled, rather than always adding it to the stack and doing lazy import on each request.
Yeah it would be slightly more efficient but it makes it a bit harder to test, bc then the middleware wouldn't be available on tests if the environment to activate it is not properly set in the host machine. A possible solution is to add a fixture that adds and removes the middleware to the |
|
yes, if the profiling feature is intended for profiling during testing using a fixture to selectively install the middleware seems like the best solution |
| # wasn't possible to send msmnt to fastpath, try to send it to s3 | ||
| ts_prefix = now.strftime("%Y%m%d%H") | ||
| s3_key = f"postcans/{ts_prefix}/{ts_prefix}_{cc}_{tn}/{msmt_uid}.post" | ||
| s3_key = f"postcans/{ts_prefix}/{ts_prefix}_{cc}_{test_name}/{msmt_uid}.post" |
There was a problem hiding this comment.
Does this impact how the s3 reprocessor will handle files?
There was a problem hiding this comment.
No, actually the tn variable didn't even exist. It was a silent bug that hasn't triggered bc the fastpath has been quite stable lately. test_name is the new name of the same variable
3 participants
Adds a profiling middleware that generates HTML reports of the specified endpoints, using pyinstrument
Some examples:
This is specially useful for diagnosing performance issues, particularly the new submission path that has been showing poor performance
Usage
We have now exposed two new environment variables:
To specify what endpoints should be profiled, use the
whitelistlist of accepted prefixes:backend/ooniapi/services/ooniprobe/src/ooniprobe/main.py
Line 96 in 7804232