Skip to content

Latest commit

 

History

History
124 lines (100 loc) · 11 KB

CONTRIBUTION_NOTES.md

File metadata and controls

124 lines (100 loc) · 11 KB

Contribution Notes: IgnitionAI Enhancements (April 2025)

This document details the enhancements and modifications made to the IgnitionAI project, specifically focusing on the r3f/target-chasing demo application. The goal was to generalize the project, add visualization capabilities, implement a user-friendly configuration interface, and provide a foundation for a visual network designer, all while keeping long-term contribution in mind.

1. Project Setup & Analysis

  • Repository Cloned: The project was cloned from https://github.com/IgnitionAI/ignition.git.
  • Dependency Installation: Dependencies were installed using pnpm install in the root directory.
  • Structure Analysis: The monorepo structure (pnpm-workspace.yaml) was identified, with key packages being @ignitionai/core, @ignitionai/backend-tfjs, and the demo application r3f/target-chasing.
  • Existing Demo: The r3f/target-chasing demo uses React, React Three Fiber (R3F), Rapier physics, and Zustand for state management to visualize a simple agent learning to reach a target in a 3D environment.

2. Core Enhancements in r3f/target-chasing

Several new components and modifications were introduced to the demo application (/home/ubuntu/ignition/r3f/target-chasing/src).

2.1. Visualization Charts (components/VisualizationCharts.tsx)

  • Purpose: To provide real-time feedback on the agent's training progress.
  • Technology: Uses the recharts library for creating interactive line charts.
  • Features:
    • Reward Chart: Displays the reward received by the agent at each step.
    • Loss Chart: Displays the training loss of the DQN agent (currently simulated, requires integration with backend-tfjs training loop).
    • Epsilon Chart: Shows the decay of the epsilon value (exploration rate) over time (currently simulated based on episode count, ideally fetched from the agent).
  • Implementation:
    • A new React component VisualizationCharts was created.
    • It uses the useTrainingStore (Zustand) to get the current reward and episodeSteps.
    • It maintains internal state (rewardHistory, lossHistory, epsilonHistory) to store data points for the charts.
    • useEffect hooks are used to update the history arrays when relevant state changes.
    • Data history is capped (maxDataPoints) to prevent performance degradation.
    • ResponsiveContainer from recharts ensures charts adapt to the panel size.
  • Future Work:
    • Integrate actual loss and epsilon values from the DQNAgent in backend-tfjs. This might require modifications to the IgnitionEnv or DQNAgent to expose these values, potentially via callbacks or updated state in the store.

2.2. Agent Configuration Panel (components/AgentConfigPanel.tsx)

  • Purpose: Allows users to modify the DQN agent's hyperparameters and network architecture without editing the code directly.
  • Technology: Standard React functional component with state management (useState).
  • Features:
    • Configure network architecture (Input Size, Action Size, Hidden Layers - add/remove/modify neuron counts).
    • Configure training parameters (Epsilon, Epsilon Decay, Min Epsilon, Gamma, Learning Rate, Batch Size, Memory Size).
    • "Apply Configuration" button triggers a callback (onApplyConfig) to pass the updated configuration to the parent component (App.tsx).
  • Implementation:
    • Uses controlled input components (<input type="number">) for each parameter.
    • State hooks manage the current value of each configuration parameter.
    • Functions addLayer, removeLayer, updateLayer manage the dynamic hidden layer configuration.
  • Integration: The App.tsx component manages the agentConfig state and passes it down to Experience.tsx. The AgentConfigPanel updates this state via the handleApplyConfig callback.

2.3. Network Designer (components/NetworkDesigner.tsx)

  • Purpose: Provides a visual, drag-and-drop interface for designing the neural network architecture. This is a foundational implementation.
  • Technology: Uses the reactflow library (v11).
    • Note: Initially react-flow-renderer (v10) was installed but replaced with reactflow as the former is deprecated.
  • Features:
    • Displays nodes (Input, Dense Layers, Output) and edges representing connections.
    • Basic interaction: Drag nodes, potentially add/remove nodes/edges (though add/remove logic is basic).
    • Extracts a simplified hidden layer structure (array of neuron counts) based on the nodes present.
    • Updates the parent component (App.tsx) via the onNetworkChange callback when the structure changes.
  • Implementation:
    • Uses ReactFlowProvider and ReactFlow components.
    • Manages elements (nodes and edges) state.
    • onConnect, onElementsRemove, onLoad callbacks handle basic interactions.
    • extractNetworkStructure function attempts to parse the hidden layer configuration from the node labels (this is a simplification).
  • Limitations & Future Work:
    • Visual Only: Currently, the designer primarily serves as a visual aid. The actual network structure used by the agent is still primarily driven by the AgentConfigPanel (specifically the hidden layer neuron counts).
    • Limited Functionality: Adding new nodes with specific types (e.g., different activation functions) or editing neuron counts directly on the nodes is not implemented.
    • Structure Extraction: The logic to translate the visual graph into a precise layer configuration (extractNetworkStructure) is basic and needs significant improvement to handle complex graphs, different layer types, and connection validation.
    • Integration: Needs deeper integration so that the visual design directly and accurately defines the agent's network architecture passed to backend-tfjs.

2.4. UI Styling (styles.css)

  • Purpose: To style the new UI panels and ensure a consistent look.
  • Changes:
    • Added a .ui-panels container to hold the control/visualization panels on the right side.
    • Added specific styles for .visualization-charts, .agent-config-panel, and .network-designer-panel.
    • Included basic styling for reactflow elements to match the dark theme.

2.5. Application Entry Point (App.tsx)

  • Purpose: Integrates all UI components and manages the overall application state related to configuration.
  • Changes:
    • Imports and renders VisualizationCharts, AgentConfigPanel, and NetworkDesigner within the .ui-panels container.
    • Manages the agentConfig state.
    • Implements handleApplyConfig and handleNetworkChange callbacks to receive updates from the child components and update the agentConfig state.
    • Passes the agentConfig state down to the Experience component.
    • Modified startTraining and resetEnvironment calls to pass the current agentConfig to the Experience component, ensuring the agent is created/reset with the latest settings.

2.6. Core Experience (Experience.tsx)

  • Purpose: Manages the 3D scene, physics, agent logic, and environment interaction.
  • Changes:
    • Modified the Agent and Experience components to accept agentConfig as a prop.
    • Created an initializeEnvironment function within the Agent component. This function:
      • Takes the agentConfig as input.
      • Disposes of the previous TFJS model (agentRefInternal.current?.dispose()) to prevent memory leaks when the configuration changes.
      • Creates a new DQNAgent instance using the provided configuration.
      • Creates a new IgnitionEnv instance with the new agent.
    • Added a useEffect hook in the Agent component that calls initializeEnvironment whenever the agentConfig prop changes. This ensures the agent and environment are recreated with the new settings.
    • Modified the startTraining and resetEnvironment functions (exposed via useImperativeHandle) to accept the agentConfig and potentially re-initialize the environment if the config has changed since the last initialization.
    • Added CuboidCollider with sensor property to the Cible component for more reliable collision detection.
    • Minor refactoring for clarity and state management using Zustand.

3. Code Generalization & Modularity

  • Dynamic Configuration: The most significant generalization was making the agent's configuration dynamic. Instead of being hardcoded in Experience.tsx, the configuration is now managed in App.tsx and driven by the UI panels (AgentConfigPanel, NetworkDesigner). This allows users to experiment with different settings without modifying the source code.
  • Component Structure: Breaking down the UI into separate components (TrainingControls, VisualizationCharts, AgentConfigPanel, NetworkDesigner) improves modularity and maintainability.
  • State Management: Continued use of Zustand (useTrainingStore) provides a centralized way to manage training-related state accessible by multiple components.

4. Roadmap Update (roadmap.md)

  • The roadmap was updated to reflect the implemented features:
    • Added visualization charts to Phase 2.
    • Added dynamic configuration, config panel, and the basic network designer to Phase 3.
    • Marked relevant items as completed (✅).
    • Added notes about the current limitations of the network designer.

5. Considerations for Long-Term Contribution

  • React Flow Integration: The current Network Designer is basic. A key next step is to implement robust logic to translate the visual graph from React Flow into a valid network configuration (potentially a sequence of layer definitions) that can be directly used by @ignitionai/backend-tfjs. This involves defining custom node types, handling connections properly, and potentially adding UI elements for configuring layer parameters directly on the nodes.
  • Backend Integration (Loss/Epsilon): The visualization charts need access to real-time loss and epsilon values from the backend. This requires exposing these metrics from the DQNAgent training loop, possibly through callbacks passed during environment creation or by updating the Zustand store from within the backend package (which might be less ideal due to coupling).
  • Performance: While basic optimizations weren't the focus of this contribution, potential bottlenecks could arise from frequent state updates for charts or complex React Flow graphs. Performance profiling might be needed later.
  • Error Handling: More robust error handling could be added, especially around agent creation with potentially invalid configurations from the UI.
  • Code Comments: Added basic comments to new components, but further commenting, especially in complex logic areas (like environment interaction or future network graph parsing), would be beneficial.

This contribution provides a significant step towards a more user-friendly and configurable interface for the IgnitionAI framework, laying the groundwork for further enhancements in visualization and no-code network design.