This is the refactored, modular version of the AppDynamics Data Extractor. The original monolithic 1,697-line file has been restructured into a clean, maintainable, and testable codebase.
AppDynamics-Data-Extractor/
├── config/ # Configuration management
│ ├── __init__.py
│ ├── settings.py # Application settings and configuration
│ └── secrets_manager.py # API credentials management
├── auth/ # Authentication
│ ├── __init__.py
│ └── appd_auth.py # AppDynamics authentication
├── api/ # API client
│ ├── __init__.py
│ └── appd_client.py # REST API client
├── utils/ # Utilities
│ ├── __init__.py
│ ├── validators.py # Data validation utilities
│ └── data_processors.py # Data processing utilities
├── data_processing/ # Data extraction and processing
│ ├── __init__.py
│ └── extractor.py # Main data extraction logic
├── ui/ # User interface
│ ├── __init__.py
│ └── components.py # Streamlit UI components
├── main.py # Main application entry point
├── appd-extractor.py # Original monolithic file (kept for reference)
└── requirements.txt
- Configuration: Centralized settings and secrets management
- Authentication: Isolated authentication logic
- API Layer: Clean REST API client
- Data Processing: Business logic separated from UI
- UI Components: Reusable Streamlit components
- Each module has a single responsibility
- Easy to locate and modify specific functionality
- Clear interfaces between modules
- Reduced code duplication
- Each module can be unit tested independently
- Mock dependencies easily
- Isolated business logic from UI concerns
- Easy to add new features
- Modular design allows for easy extension
- Clear dependency management
- Type hints throughout
- Docstrings for all functions
- Consistent error handling
- Clean imports and dependencies
- Before: Global variables scattered throughout the code
- After: Centralized
AppConfigclass with type hints and environment variable support
- Before: Mixed authentication logic with API calls
- After: Dedicated
AppDAuthenticatorclass with token management
- Before: Individual functions for each API call
- After: Unified
AppDAPIClientclass with consistent error handling
- Before: Mixed data processing with UI logic
- After: Dedicated
AppDDataExtractorclass with clear data flow
- Before: Inline UI code mixed with business logic
- After: Reusable UI components with clear interfaces
Manages application configuration and secrets:
settings.py: Application settings with type hintssecrets_manager.py: YAML-based secrets management
Handles AppDynamics authentication:
appd_auth.py: OAuth token management and session handling
REST API client for AppDynamics:
appd_client.py: Unified API client with error handling
Common utilities and helpers:
validators.py: Data validation and JSON/XML parsingdata_processors.py: Data transformation and license calculations
Main data extraction and processing:
extractor.py: Orchestrates data extraction and processing
Streamlit UI components:
components.py: Reusable UI components
- Python 3.8+
- All dependencies from
requirements.txt
-
Using the new modular version:
streamlit run main.py
-
Using the original version (for comparison):
streamlit run appd-extractor.py
The modular structure makes development much easier:
# Example: Adding a new API endpoint
from api import AppDAPIClient
from auth import AppDAuthenticator
from config import APICredentials
# Create credentials
credentials = APICredentials(
account_name="your-account",
api_client="your-client",
api_secret="your-secret"
)
# Authenticate
authenticator = AppDAuthenticator(credentials)
if authenticator.authenticate():
# Use API client
api_client = AppDAPIClient(authenticator)
response, status = api_client.get_applications()Each module can be tested independently:
# Example test for API client
import unittest
from unittest.mock import Mock, patch
from api import AppDAPIClient
class TestAppDAPIClient(unittest.TestCase):
def setUp(self):
self.mock_authenticator = Mock()
self.api_client = AppDAPIClient(self.mock_authenticator)
def test_get_applications(self):
# Test implementation
pass- Configuration: Replace global variables with
AppConfiginstance - Authentication: Use
AppDAuthenticatorinstead of inline auth logic - API Calls: Use
AppDAPIClientmethods instead of individual functions - Data Processing: Use
AppDDataExtractorfor data extraction - UI: Use UI components from
ui/components.py
The original appd-extractor.py is preserved for reference and can still be used. The modular version provides the same functionality with improved architecture.
- Memory Management: Better garbage collection and memory cleanup
- Error Handling: Consistent error handling across all modules
- Token Management: Automatic token refresh and validation
- Data Processing: Optimized data merging and processing
The modular structure makes it easy to add:
- New Data Sources: Add new API endpoints in
api/appd_client.py - Additional Processors: Add new data processors in
utils/data_processors.py - UI Improvements: Add new UI components in
ui/components.py - Configuration Options: Extend
config/settings.py - Authentication Methods: Add new auth methods in
auth/
- Type Hints: Full type annotation support
- Docstrings: Comprehensive documentation
- Error Handling: Consistent error handling patterns
- Logging: Structured logging (can be easily added)
- Testing: Testable architecture
This modular structure transforms the original 1,697-line monolithic file into a clean, maintainable, and extensible codebase that follows Python best practices and software engineering principles.