Skip to content

Auto-generate low/high level AWS wrappers#121

Merged
mattBrzezinski merged 13 commits into
v1from
MB/service-generation
Apr 8, 2020
Merged

Auto-generate low/high level AWS wrappers#121
mattBrzezinski merged 13 commits into
v1from
MB/service-generation

Conversation

@mattBrzezinski

Copy link
Copy Markdown
Member

Overview

This merge request generates the low and high level wrappers for AWS Services in Julia, and is the initial MR for the new state of AWS in Julia. The design document for this can be found at, mattBrzezinski/JuliaCloud-AWS-State.

Note: This is targeting a feature branch v1 where we will build up the new version before releasing it.

Components

  • src/AWS.jl:
    • Contains the structure for each type of request type (wraps around AWSCore.jl currently)
    • Macro for including a high-level wrapper (see test/AWS.jl for a use case)
  • src/AWSMetadata.jl:
    • Higher level functions for generating low and high level wrappers
  • src/AWSMetadataUtilities.jl:
    • Utility functions used in AWSMetadata.jl
  • src/AWSServices.jl:
    • Auto-generated low-level wrapper for AWS Services
  • services/*:
    • Auto-generated high-level wrappers for AWS Services
  • metadata.json:
    • Stores the AWS services .normal.json filename, sha hash, and version for knowing when to update or create new wrappers

How to review this merge request

The best way to review this merge request would be to look at each commit individually. Skip commit 2af1ec824 as it is only auto-generated files.

Next steps towards a better AWS world

We need to port over AWSCredentials.jl, AWSConfig.jl, and begin to re-write the functions which make the requests themselves (along with utilities for them):

# Returns
- `latest_versions::Dict[]`: List of the latest AWS Service definitions
"""
function _filter_latest_service_version(services::Array)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this could be done better, but I spent some time on it but wasn't able to use list comprehension for this and decided to make it more explicit.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's pretty common to drop the type assertion and just expect the input is iterable

@mattBrzezinski

Copy link
Copy Markdown
Member Author

Some of the docstrings need to be updated, especially for the input/output types, I'll be re-working these.

Comment thread Project.toml
Comment thread src/AWS.jl Outdated
Comment thread src/AWS.jl Outdated
Comment thread src/AWSMetadata.jl Outdated
Comment thread src/AWSMetadataUtilities.jl Outdated
Comment thread src/AWSMetadataUtilities.jl Outdated
Comment thread src/AWSMetadata.jl Outdated
Comment thread src/AWSMetadata.jl Outdated
Comment thread src/AWSMetadata.jl Outdated
Comment thread src/AWSMetadata.jl
$(join(service_definitions, "\n"))

end
"""

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be easier to read if you write this like this:

    template = """
        # This file is auto-generated by AWSMetadata.jl
        module AWSServices
        include("AWS.jl")
        $(join(service_definitions, "\n"))
        end
        """

# Returns
- `latest_versions::Dict[]`: List of the latest AWS Service definitions
"""
function _filter_latest_service_version(services::Array)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's pretty common to drop the type assertion and just expect the input is iterable

Comment thread src/AWSMetadataUtilities.jl Outdated
Comment thread src/AWSMetadataUtilities.jl Outdated
Comment thread src/AWSMetadataUtilities.jl Outdated
Comment thread src/services/accessanalyzer.jl
Comment thread src/services/accessanalyzer.jl
@mattBrzezinski
mattBrzezinski force-pushed the MB/service-generation branch from a3c4699 to 169254e Compare April 2, 2020 17:23
- Removed dependency on DataStructures in favour of OrderedCollections
- Moved `src/metadata.json` to `deps/metadata.json`
- Added comment stating that files in `src/services/*` are
  auto-generated
@mattBrzezinski
mattBrzezinski force-pushed the MB/service-generation branch from 169254e to d6166d1 Compare April 2, 2020 17:24
Comment thread Project.toml Outdated
Comment thread src/AWSMetadataUtilities.jl
Comment thread src/AWS.jl Outdated
@mattBrzezinski
mattBrzezinski force-pushed the MB/service-generation branch from fe3cd9f to d04d976 Compare April 2, 2020 21:30
@mattBrzezinski
mattBrzezinski force-pushed the MB/service-generation branch from d04d976 to f1292ae Compare April 2, 2020 21:39

@omus omus left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looking good. Did one more pass and found some minor things.

Comment thread src/AWS.jl Outdated
Comment thread src/AWSExceptions.jl Outdated
Comment thread src/AWSExceptions.jl Outdated
Comment thread src/AWSMetadata.jl Outdated
Comment thread src/AWSMetadata.jl Outdated
Comment thread src/AWSMetadata.jl Outdated
Comment thread src/AWSMetadata.jl Outdated
Comment thread src/AWSMetadataUtilities.jl Outdated
Comment thread src/AWSMetadataUtilities.jl
Comment thread src/AWSMetadataUtilities.jl Outdated
@mattBrzezinski
mattBrzezinski merged commit f321d8c into v1 Apr 8, 2020
@omus
omus deleted the MB/service-generation branch April 9, 2020 19:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants