rfc: Migrate most configs from setup.py to pyproject.toml

[YvanY0]

Call python setup.py directly is deprecated, to modernize this
project, let's introduce pyproject.toml, it supports more features. For
compatibility with the past, keep setup.py in present.

Signed-off-by: Yihuang Yu yihyu@redhat.com

Summary by CodeRabbit

• New Features

  • Added Makefile targets to enable developing all or specific optional plugins.
  • Introduced a new remote-host spawner with slot reservation for remote task execution.

• Bug Fixes

  • Improved Robot plugin test discovery to support newer test case APIs.

• Documentation

  • Updated contributor and plugin guides to use pyproject.toml and pip workflows; added plugin READMEs (Ansible, others).

• Chores

  • Migrated CI/workflows and developer tooling from setup.py to pip/module-based commands; added pyproject.toml manifests across core and optional plugins.

• Tests

  • Simplified unit tests to use direct queue-based invocation for software manager paths.