This document details the enhancements and modifications made to the IgnitionAI project, specifically focusing on the r3f/target-chasing demo application. The goal was to generalize the project, add visualization capabilities, implement a user-friendly configuration interface, and provide a foundation for a visual network designer, all while keeping long-term contribution in mind.
- Repository Cloned: The project was cloned from
https://github.com/IgnitionAI/ignition.git. - Dependency Installation: Dependencies were installed using
pnpm installin the root directory. - Structure Analysis: The monorepo structure (
pnpm-workspace.yaml) was identified, with key packages being@ignitionai/core,@ignitionai/backend-tfjs, and the demo applicationr3f/target-chasing. - Existing Demo: The
r3f/target-chasingdemo uses React, React Three Fiber (R3F), Rapier physics, and Zustand for state management to visualize a simple agent learning to reach a target in a 3D environment.
Several new components and modifications were introduced to the demo application (/home/ubuntu/ignition/r3f/target-chasing/src).
- Purpose: To provide real-time feedback on the agent's training progress.
- Technology: Uses the
rechartslibrary for creating interactive line charts. - Features:
- Reward Chart: Displays the reward received by the agent at each step.
- Loss Chart: Displays the training loss of the DQN agent (currently simulated, requires integration with
backend-tfjstraining loop). - Epsilon Chart: Shows the decay of the epsilon value (exploration rate) over time (currently simulated based on episode count, ideally fetched from the agent).
- Implementation:
- A new React component
VisualizationChartswas created. - It uses the
useTrainingStore(Zustand) to get the currentrewardandepisodeSteps. - It maintains internal state (
rewardHistory,lossHistory,epsilonHistory) to store data points for the charts. useEffecthooks are used to update the history arrays when relevant state changes.- Data history is capped (
maxDataPoints) to prevent performance degradation. ResponsiveContainerfromrechartsensures charts adapt to the panel size.
- A new React component
- Future Work:
- Integrate actual loss and epsilon values from the
DQNAgentinbackend-tfjs. This might require modifications to theIgnitionEnvorDQNAgentto expose these values, potentially via callbacks or updated state in the store.
- Integrate actual loss and epsilon values from the
- Purpose: Allows users to modify the DQN agent's hyperparameters and network architecture without editing the code directly.
- Technology: Standard React functional component with state management (
useState). - Features:
- Configure network architecture (Input Size, Action Size, Hidden Layers - add/remove/modify neuron counts).
- Configure training parameters (Epsilon, Epsilon Decay, Min Epsilon, Gamma, Learning Rate, Batch Size, Memory Size).
- "Apply Configuration" button triggers a callback (
onApplyConfig) to pass the updated configuration to the parent component (App.tsx).
- Implementation:
- Uses controlled input components (
<input type="number">) for each parameter. - State hooks manage the current value of each configuration parameter.
- Functions
addLayer,removeLayer,updateLayermanage the dynamic hidden layer configuration.
- Uses controlled input components (
- Integration: The
App.tsxcomponent manages theagentConfigstate and passes it down toExperience.tsx. TheAgentConfigPanelupdates this state via thehandleApplyConfigcallback.
- Purpose: Provides a visual, drag-and-drop interface for designing the neural network architecture. This is a foundational implementation.
- Technology: Uses the
reactflowlibrary (v11).- Note: Initially
react-flow-renderer(v10) was installed but replaced withreactflowas the former is deprecated.
- Note: Initially
- Features:
- Displays nodes (Input, Dense Layers, Output) and edges representing connections.
- Basic interaction: Drag nodes, potentially add/remove nodes/edges (though add/remove logic is basic).
- Extracts a simplified hidden layer structure (array of neuron counts) based on the nodes present.
- Updates the parent component (
App.tsx) via theonNetworkChangecallback when the structure changes.
- Implementation:
- Uses
ReactFlowProviderandReactFlowcomponents. - Manages
elements(nodes and edges) state. onConnect,onElementsRemove,onLoadcallbacks handle basic interactions.extractNetworkStructurefunction attempts to parse the hidden layer configuration from the node labels (this is a simplification).
- Uses
- Limitations & Future Work:
- Visual Only: Currently, the designer primarily serves as a visual aid. The actual network structure used by the agent is still primarily driven by the
AgentConfigPanel(specifically the hidden layer neuron counts). - Limited Functionality: Adding new nodes with specific types (e.g., different activation functions) or editing neuron counts directly on the nodes is not implemented.
- Structure Extraction: The logic to translate the visual graph into a precise layer configuration (
extractNetworkStructure) is basic and needs significant improvement to handle complex graphs, different layer types, and connection validation. - Integration: Needs deeper integration so that the visual design directly and accurately defines the agent's network architecture passed to
backend-tfjs.
- Visual Only: Currently, the designer primarily serves as a visual aid. The actual network structure used by the agent is still primarily driven by the
- Purpose: To style the new UI panels and ensure a consistent look.
- Changes:
- Added a
.ui-panelscontainer to hold the control/visualization panels on the right side. - Added specific styles for
.visualization-charts,.agent-config-panel, and.network-designer-panel. - Included basic styling for
reactflowelements to match the dark theme.
- Added a
- Purpose: Integrates all UI components and manages the overall application state related to configuration.
- Changes:
- Imports and renders
VisualizationCharts,AgentConfigPanel, andNetworkDesignerwithin the.ui-panelscontainer. - Manages the
agentConfigstate. - Implements
handleApplyConfigandhandleNetworkChangecallbacks to receive updates from the child components and update theagentConfigstate. - Passes the
agentConfigstate down to theExperiencecomponent. - Modified
startTrainingandresetEnvironmentcalls to pass the currentagentConfigto theExperiencecomponent, ensuring the agent is created/reset with the latest settings.
- Imports and renders
- Purpose: Manages the 3D scene, physics, agent logic, and environment interaction.
- Changes:
- Modified the
AgentandExperiencecomponents to acceptagentConfigas a prop. - Created an
initializeEnvironmentfunction within theAgentcomponent. This function:- Takes the
agentConfigas input. - Disposes of the previous TFJS model (
agentRefInternal.current?.dispose()) to prevent memory leaks when the configuration changes. - Creates a new
DQNAgentinstance using the provided configuration. - Creates a new
IgnitionEnvinstance with the new agent.
- Takes the
- Added a
useEffecthook in theAgentcomponent that callsinitializeEnvironmentwhenever theagentConfigprop changes. This ensures the agent and environment are recreated with the new settings. - Modified the
startTrainingandresetEnvironmentfunctions (exposed viauseImperativeHandle) to accept theagentConfigand potentially re-initialize the environment if the config has changed since the last initialization. - Added
CuboidColliderwithsensorproperty to theCiblecomponent for more reliable collision detection. - Minor refactoring for clarity and state management using Zustand.
- Modified the
- Dynamic Configuration: The most significant generalization was making the agent's configuration dynamic. Instead of being hardcoded in
Experience.tsx, the configuration is now managed inApp.tsxand driven by the UI panels (AgentConfigPanel,NetworkDesigner). This allows users to experiment with different settings without modifying the source code. - Component Structure: Breaking down the UI into separate components (
TrainingControls,VisualizationCharts,AgentConfigPanel,NetworkDesigner) improves modularity and maintainability. - State Management: Continued use of Zustand (
useTrainingStore) provides a centralized way to manage training-related state accessible by multiple components.
- The roadmap was updated to reflect the implemented features:
- Added visualization charts to Phase 2.
- Added dynamic configuration, config panel, and the basic network designer to Phase 3.
- Marked relevant items as completed (✅).
- Added notes about the current limitations of the network designer.
- React Flow Integration: The current Network Designer is basic. A key next step is to implement robust logic to translate the visual graph from React Flow into a valid network configuration (potentially a sequence of layer definitions) that can be directly used by
@ignitionai/backend-tfjs. This involves defining custom node types, handling connections properly, and potentially adding UI elements for configuring layer parameters directly on the nodes. - Backend Integration (Loss/Epsilon): The visualization charts need access to real-time loss and epsilon values from the backend. This requires exposing these metrics from the
DQNAgenttraining loop, possibly through callbacks passed during environment creation or by updating the Zustand store from within the backend package (which might be less ideal due to coupling). - Performance: While basic optimizations weren't the focus of this contribution, potential bottlenecks could arise from frequent state updates for charts or complex React Flow graphs. Performance profiling might be needed later.
- Error Handling: More robust error handling could be added, especially around agent creation with potentially invalid configurations from the UI.
- Code Comments: Added basic comments to new components, but further commenting, especially in complex logic areas (like environment interaction or future network graph parsing), would be beneficial.
This contribution provides a significant step towards a more user-friendly and configurable interface for the IgnitionAI framework, laying the groundwork for further enhancements in visualization and no-code network design.