This repository was archived by the owner on Nov 26, 2020. It is now read-only.
Repository navigation
This repository was archived by the owner on Nov 26, 2020. It is now read-only.
Jupyter notebooks integration in Testground #8
Description
Context
The gossipsub security hardening project created a bespoke Jupyter notebooks integration that allowed users to execute test runs on cluster:k8s and analyse the outputs from within the comfort of an interactive Jupypter notebook session. It was hugely useful and successful.
We need the same for Project Oni.
Output
- Generalise the solution, decouple it from the gossipsub project, package it so that it's project-agnostic and reusable.
- Contribute it to
testground/testground. - Based on conversations with Yusef, the main point to figure out is how to automatically translate the test parameters as expressed in the manifest schema into Jupyter widgets.
One proposal was to use a JSON schema => Jupyter Widgets library, by previously translating our super-simple manifest definitions to basic JSON schema (we really want to stay away from JSON schema in Testground).
- Allow the user to easily process test outputs from within Jupyter Notebooks.
- Tackle default parameters for composition groups.
- For
cluster:k8s, mount NFS into Jupyter Notebooks server so we can skip the collect outputs step.
Future
- pandas workers.
Activity
[-]Contribute Jupyter notebooks integration to Testground[/-][+]Jupyter notebooks integration in Testground[/+]raulk commentedon Jun 15, 2020
@yusefnapora ^^ scoping by groups.
yusefnapora commentedon Jun 15, 2020
just dropping a link to Anton's draft infra PR: testground/infra#56
yusefnapora commentedon Jun 17, 2020
I've been prototyping this using the holoviz Param library, and have some basic UI generation working.
Here's what I think we need to get working to make this functional:
Things that are nice to have but may get pushed to next week:
There's also the "other half" of this story, which is collecting and processing test outputs; I'll open a new issue to track that soon.
yusefnapora commentedon Jun 17, 2020
Also, I've been working with a local "scrach" repo; where should this code actually live? Will it belong in the main testground repo, or should we make it a standalone tool? @raulk ?
nonsense commentedon Jun 18, 2020
@yusefnapora any chance we can see a demo of this today or on Friday? I think it will help if we start small with the UI, integrate it with the infrastructure, and continue from there to refine and add features.
I can't say where the code should live, because I am not sure how much of it is testplan specific, and how much of it should be deployed alongside the infra stuff (Jupyter Hub, etc.) - I guess we can decide when we see the first iteration in action? WDYT?
raulk commentedon Jun 18, 2020
@yusefnapora it would be awesome if you could demo a sneak peek of this today (not excluding a demo on Friday!), as well as walking us through the codebase. Then @nonsense and I can decide where these components will be hosted.
raulk commentedon Jun 19, 2020
@yusefnapora as discussed yesterday, outstanding items for this sprint include:
Next sprint:
Anything that I'm missing?
yusefnapora commentedon Jun 19, 2020
@raulk I think that basically covers it.
I think that "run Jupyter notebook reliably locally" should include some kind of Docker magic to avoid dep hell.
I spun my wheels a bit on that yesterday - it's more annoying than I thought, because you can't really bind mount $TESTGROUND_HOME into a container and access plan manifests that were symlink'd in with
testground plan import, since the symlinks aren't traversable inside the container.So I may punt on that for right now and require a python virtualenv to run locally, or just force the user to manually provide the manifest file. The latter might be simpler and less "magic" anyhow, so maybe we should just do that.
nonsense commentedon Jun 19, 2020
We should definitely do that - Docker is more-or-less a dependency for Testground today, so no reason to not use it and have users install manually Python and related dependencies.