Skip to content

Latest commit

 

History

History
222 lines (168 loc) · 6.95 KB

README_MODULAR.md

File metadata and controls

222 lines (168 loc) · 6.95 KB

AppDynamics Data Extractor - Modular Version

This is the refactored, modular version of the AppDynamics Data Extractor. The original monolithic 1,697-line file has been restructured into a clean, maintainable, and testable codebase.

🏗️ New Architecture

Directory Structure

AppDynamics-Data-Extractor/
├── config/                 # Configuration management
│   ├── __init__.py
│   ├── settings.py         # Application settings and configuration
│   └── secrets_manager.py  # API credentials management
├── auth/                   # Authentication
│   ├── __init__.py
│   └── appd_auth.py       # AppDynamics authentication
├── api/                    # API client
│   ├── __init__.py
│   └── appd_client.py     # REST API client
├── utils/                  # Utilities
│   ├── __init__.py
│   ├── validators.py      # Data validation utilities
│   └── data_processors.py # Data processing utilities
├── data_processing/        # Data extraction and processing
│   ├── __init__.py
│   └── extractor.py       # Main data extraction logic
├── ui/                     # User interface
│   ├── __init__.py
│   └── components.py      # Streamlit UI components
├── main.py                # Main application entry point
├── appd-extractor.py      # Original monolithic file (kept for reference)
└── requirements.txt

🚀 Benefits of the New Structure

1. Separation of Concerns

  • Configuration: Centralized settings and secrets management
  • Authentication: Isolated authentication logic
  • API Layer: Clean REST API client
  • Data Processing: Business logic separated from UI
  • UI Components: Reusable Streamlit components

2. Maintainability

  • Each module has a single responsibility
  • Easy to locate and modify specific functionality
  • Clear interfaces between modules
  • Reduced code duplication

3. Testability

  • Each module can be unit tested independently
  • Mock dependencies easily
  • Isolated business logic from UI concerns

4. Scalability

  • Easy to add new features
  • Modular design allows for easy extension
  • Clear dependency management

5. Code Quality

  • Type hints throughout
  • Docstrings for all functions
  • Consistent error handling
  • Clean imports and dependencies

🔧 Key Improvements

Configuration Management

  • Before: Global variables scattered throughout the code
  • After: Centralized AppConfig class with type hints and environment variable support

Authentication

  • Before: Mixed authentication logic with API calls
  • After: Dedicated AppDAuthenticator class with token management

API Client

  • Before: Individual functions for each API call
  • After: Unified AppDAPIClient class with consistent error handling

Data Processing

  • Before: Mixed data processing with UI logic
  • After: Dedicated AppDDataExtractor class with clear data flow

UI Components

  • Before: Inline UI code mixed with business logic
  • After: Reusable UI components with clear interfaces

📦 Module Descriptions

config/

Manages application configuration and secrets:

  • settings.py: Application settings with type hints
  • secrets_manager.py: YAML-based secrets management

auth/

Handles AppDynamics authentication:

  • appd_auth.py: OAuth token management and session handling

api/

REST API client for AppDynamics:

  • appd_client.py: Unified API client with error handling

utils/

Common utilities and helpers:

  • validators.py: Data validation and JSON/XML parsing
  • data_processors.py: Data transformation and license calculations

data_processing/

Main data extraction and processing:

  • extractor.py: Orchestrates data extraction and processing

ui/

Streamlit UI components:

  • components.py: Reusable UI components

🚀 Getting Started

Prerequisites

  • Python 3.8+
  • All dependencies from requirements.txt

Running the Application

  1. Using the new modular version:

    streamlit run main.py
  2. Using the original version (for comparison):

    streamlit run appd-extractor.py

Development

The modular structure makes development much easier:

# Example: Adding a new API endpoint
from api import AppDAPIClient
from auth import AppDAuthenticator
from config import APICredentials

# Create credentials
credentials = APICredentials(
    account_name="your-account",
    api_client="your-client",
    api_secret="your-secret"
)

# Authenticate
authenticator = AppDAuthenticator(credentials)
if authenticator.authenticate():
    # Use API client
    api_client = AppDAPIClient(authenticator)
    response, status = api_client.get_applications()

🧪 Testing

Each module can be tested independently:

# Example test for API client
import unittest
from unittest.mock import Mock, patch
from api import AppDAPIClient

class TestAppDAPIClient(unittest.TestCase):
    def setUp(self):
        self.mock_authenticator = Mock()
        self.api_client = AppDAPIClient(self.mock_authenticator)
    
    def test_get_applications(self):
        # Test implementation
        pass

🔄 Migration Guide

From Monolithic to Modular

  1. Configuration: Replace global variables with AppConfig instance
  2. Authentication: Use AppDAuthenticator instead of inline auth logic
  3. API Calls: Use AppDAPIClient methods instead of individual functions
  4. Data Processing: Use AppDDataExtractor for data extraction
  5. UI: Use UI components from ui/components.py

Backward Compatibility

The original appd-extractor.py is preserved for reference and can still be used. The modular version provides the same functionality with improved architecture.

📈 Performance Improvements

  • Memory Management: Better garbage collection and memory cleanup
  • Error Handling: Consistent error handling across all modules
  • Token Management: Automatic token refresh and validation
  • Data Processing: Optimized data merging and processing

🛠️ Future Enhancements

The modular structure makes it easy to add:

  1. New Data Sources: Add new API endpoints in api/appd_client.py
  2. Additional Processors: Add new data processors in utils/data_processors.py
  3. UI Improvements: Add new UI components in ui/components.py
  4. Configuration Options: Extend config/settings.py
  5. Authentication Methods: Add new auth methods in auth/

📝 Code Quality

  • Type Hints: Full type annotation support
  • Docstrings: Comprehensive documentation
  • Error Handling: Consistent error handling patterns
  • Logging: Structured logging (can be easily added)
  • Testing: Testable architecture

This modular structure transforms the original 1,697-line monolithic file into a clean, maintainable, and extensible codebase that follows Python best practices and software engineering principles.