Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions docs/tutorials/predictor-tabular.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ cloud_predictor.fit(
`train_data` can be a pandas DataFrame, or a path to a local or S3 file (CSV or Parquet). In every case AutoGluon-Cloud loads the data locally and uploads it to your `cloud_output_path` bucket before kicking off the SageMaker job.

### Reattach to a training job
If your local connection drops, the training job keeps running on SageMaker. You can reattach with another `CloudPredictor` via {py:meth}`~autogluon.cloud.TabularCloudPredictor.attach_job` as long as you have the job name — it's logged when training starts (`INFO:sagemaker:Creating training-job with name: ag-cloudpredictor-...`) and also visible in the SageMaker console.
If your local connection drops, the training job keeps running on SageMaker. You can reattach with another `CloudPredictor` via {py:meth}`~autogluon.cloud.TabularCloudPredictor.attach_job` as long as you have the job name — it's logged when training starts (`INFO:sagemaker:Creating training-job with name: ag-cloud-tabular-...`) and also visible in the SageMaker console.

```python
another_cloud_predictor = TabularCloudPredictor()
Expand Down Expand Up @@ -165,18 +165,18 @@ It will output a dict similar to this:
'local_output_path': '/home/ubuntu/XXX/demo/AutogluonCloudPredictor/ag-20221111_174928',
'cloud_output_path': 's3://XXX/tabular-demo',
'fit_job': {
'name': 'ag-cloudpredictor-1668188968-e5c3',
'name': 'ag-cloud-tabular-1668188968-e5c3',
'status': 'Completed',
'framework_version': '0.6.1',
'artifact_path': 's3://XXX/tabular-demo/model/ag-cloudpredictor-1668188968-e5c3/output/model.tar.gz'
'artifact_path': 's3://XXX/tabular-demo/model/ag-cloud-tabular-1668188968-e5c3/output/model.tar.gz'
},
'recent_transform_job': {
'name': 'ag-cloudpredictor-1668189393-e95c',
'name': 'ag-cloud-tabular-1668189393-e95c',
'status': 'Completed',
'result_path': 's3://XXX/tabular-demo/batch_transform/2022-11-11-17-56-33-991/results/test.csv.out'
},
'transform_jobs': ['ag-cloudpredictor-1668189393-e95c'],
'endpoint': 'ag-cloudpredictor-1668189208-d23b'
'transform_jobs': ['ag-cloud-tabular-1668189393-e95c'],
'endpoint': 'ag-cloud-tabular-1668189208-d23b'
}
```

Expand Down
12 changes: 6 additions & 6 deletions docs/tutorials/predictor-timeseries.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ forecasts = cloud_predictor.fit_predict(
By default predictions land at `{cloud_output_path}/{job_name}/predictions.csv`; pass `predictions_path` to choose a destination.

### Reattach to a training job
If your local connection drops, the training job keeps running on SageMaker. You can reattach with another `CloudPredictor` via {py:meth}`~autogluon.cloud.TimeSeriesCloudPredictor.attach_job` as long as you have the job name — it's logged when training starts (`INFO:sagemaker:Creating training-job with name: ag-cloudpredictor-...`) and also visible in the SageMaker console.
If your local connection drops, the training job keeps running on SageMaker. You can reattach with another `CloudPredictor` via {py:meth}`~autogluon.cloud.TimeSeriesCloudPredictor.attach_job` as long as you have the job name — it's logged when training starts (`INFO:sagemaker:Creating training-job with name: ag-cloud-timeseries-...`) and also visible in the SageMaker console.

```python
another_cloud_predictor = TimeSeriesCloudPredictor()
Expand Down Expand Up @@ -250,18 +250,18 @@ It will output a dict similar to this:
'local_output_path': '/home/ubuntu/XXX/demo/AutogluonCloudPredictor/ag-20221111_174928',
'cloud_output_path': 's3://XXX/timeseries-demo',
'fit_job': {
'name': 'ag-cloudpredictor-1668188968-e5c3',
'name': 'ag-cloud-timeseries-1668188968-e5c3',
'status': 'Completed',
'framework_version': '0.6.1',
'artifact_path': 's3://XXX/timeseries-demo/model/ag-cloudpredictor-1668188968-e5c3/output/model.tar.gz'
'artifact_path': 's3://XXX/timeseries-demo/model/ag-cloud-timeseries-1668188968-e5c3/output/model.tar.gz'
},
'recent_transform_job': {
'name': 'ag-cloudpredictor-1668189393-e95c',
'name': 'ag-cloud-timeseries-1668189393-e95c',
'status': 'Completed',
'result_path': 's3://XXX/timeseries-demo/batch_transform/2022-11-11-17-56-33-991/results/test.parquet.out'
},
'transform_jobs': ['ag-cloudpredictor-1668189393-e95c'],
'endpoint': 'ag-cloudpredictor-1668189208-d23b'
'transform_jobs': ['ag-cloud-timeseries-1668189393-e95c'],
'endpoint': 'ag-cloud-timeseries-1668189208-d23b'
}
```

Expand Down
8 changes: 7 additions & 1 deletion src/autogluon/cloud/backend/backend.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,18 @@ def cloud_output_path(self) -> str:
return self._cloud_output_path

def initialize(
self, local_output_path: str, predictor_type: str, cloud_output_path: Optional[str] = None, **kwargs
self,
local_output_path: str,
predictor_type: str,
cloud_output_path: Optional[str] = None,
resource_prefix: Optional[str] = None,
**kwargs,
) -> None:
"""Initialize the backend."""
self.local_output_path = local_output_path
self._cloud_output_path = cloud_output_path
self.predictor_type = predictor_type
self.resource_prefix = resource_prefix or f"ag-cloud-{predictor_type}"
self.original_features = None
self.endpoint: Optional[Endpoint] = None

Expand Down
5 changes: 2 additions & 3 deletions src/autogluon/cloud/backend/ray_backend.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@
from ..endpoint.endpoint import Endpoint
from ..job.ray_job import RayFitJob
from ..scripts import ScriptManager
from ..utils.constants import CLOUD_RESOURCE_PREFIX
from ..utils.dlc_utils import parse_framework_version, retrieve_image_uri
from ..utils.ec2 import get_latest_ami
from ..utils.iam import get_instance_profile_arn
Expand Down Expand Up @@ -143,7 +142,7 @@ def fit(
If `custom_image_uri` is set, this argument will be ignored.
job_name: str, default = None
Name of the launched training job.
If None, CloudPredictor will create one with prefix ag-cloudpredictor
If None, AutoGluon Cloud creates one with a predictor-specific prefix.
instance_type: str, default = 'ml.m5.2xlarge'
Instance type the predictor will be trained on with SageMaker.
instance_count: Union[int, str], default = "auto",
Expand Down Expand Up @@ -257,7 +256,7 @@ def fit(
cluster_manager.setup_connection()
time.sleep(10) # waiting for connection to setup
if job_name is None:
job_name = CLOUD_RESOURCE_PREFIX + "-" + get_utc_timestamp_now()
job_name = self.resource_prefix + "-" + get_utc_timestamp_now()
job = RayFitJob(output_path=self.cloud_output_path + "/model")
self._fit_job = job

Expand Down
18 changes: 9 additions & 9 deletions src/autogluon/cloud/backend/sagemaker_backend.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
AutoGluonRepackInferenceModel,
)
from ..utils.aws_utils import resolve_execution_role, setup_sagemaker_session
from ..utils.constants import CLOUD_RESOURCE_PREFIX, VALID_ACCEPT
from ..utils.constants import VALID_ACCEPT
from ..utils.dlc_utils import infer_sagemaker_ami_version, parse_framework_version
from ..utils.misc import MostRecentInsertedOrderedDict
from ..utils.serializers import AutoGluonSerializationWrapper
Expand Down Expand Up @@ -207,7 +207,7 @@ def fit(
If `custom_image_uri` is set, this argument will be ignored.
job_name: str, default = None
Name of the launched training job.
If None, CloudPredictor will create one with prefix ag-cloudpredictor
If None, AutoGluon Cloud creates one with a predictor- or model-specific prefix.
instance_type: str, default = 'ml.m5.2xlarge'
Instance type the predictor will be trained on with SageMaker.
instance_count: int, default = 1
Expand Down Expand Up @@ -250,7 +250,7 @@ def fit(
logger.log(20, f"Training with framework_version=={framework_version}")

if not job_name:
job_name = sagemaker.utils.unique_name_from_base(CLOUD_RESOURCE_PREFIX)
job_name = sagemaker.utils.unique_name_from_base(self.resource_prefix)

if instance_count == "auto":
instance_count = 1
Expand Down Expand Up @@ -337,7 +337,7 @@ def fit(
volume_size=volume_size,
framework_version=framework_version,
py_version=py_version,
base_job_name="autogluon-cloudpredictor-train",
base_job_name=f"{self.resource_prefix}-train",
output_path=output_path,
code_location=code_location,
inputs=inputs,
Expand Down Expand Up @@ -386,7 +386,7 @@ def deploy(
If None, will deploy the most recent trained predictor trained with `fit()`.
endpoint_name: str
The endpoint name to use for the deployment.
If None, CloudPredictor will create one with prefix `ag-cloudpredictor`
If None, AutoGluon Cloud creates one with a predictor- or model-specific prefix.
framework_version: str, default = `latest`
Inference container version of autogluon.
If `latest`, will use the latest available container version.
Expand Down Expand Up @@ -433,7 +433,7 @@ def deploy(
# Needed to infer the container image (CPU vs GPU) downstream — serverless is CPU-only.
instance_type = "ml.m5.2xlarge"
if not endpoint_name:
endpoint_name = sagemaker.utils.unique_name_from_base(CLOUD_RESOURCE_PREFIX)
endpoint_name = sagemaker.utils.unique_name_from_base(self.resource_prefix)

# Resolve container image
if custom_image_uri:
Expand Down Expand Up @@ -799,7 +799,7 @@ def predict(
If `custom_image_uri` is set, this argument will be ignored.
job_name: str, default = None
Name of the launched training job.
If None, CloudPredictor will create one with prefix ag-cloudpredictor.
If None, AutoGluon Cloud creates one with a predictor- or model-specific prefix.
instance_count: int, default = 1,
Number of instances used to do batch transform.
instance_type: str, default = 'ml.m5.2xlarge'
Expand Down Expand Up @@ -904,7 +904,7 @@ def predict_proba(
If `custom_image_uri` is set, this argument will be ignored.
job_name: str, default = None
Name of the launched training job.
If None, CloudPredictor will create one with prefix ag-cloudpredictor.
If None, AutoGluon Cloud creates one with a predictor- or model-specific prefix.
instance_count: int, default = 1,
Number of instances used to do batch transform.
instance_type: str, default = 'ml.m5.2xlarge'
Expand Down Expand Up @@ -1257,7 +1257,7 @@ def _predict(
predictor_path = self._upload_predictor(predictor_path, cloud_key_prefix + "/predictor")

if not job_name:
job_name = sagemaker.utils.unique_name_from_base(CLOUD_RESOURCE_PREFIX)
job_name = sagemaker.utils.unique_name_from_base(self.resource_prefix)

if test_data_image_column is not None:
logger.warning("Batch inference with image modality could be slow because of some technical details.")
Expand Down
1 change: 1 addition & 0 deletions src/autogluon/cloud/model/foundation_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,7 @@ def __init__(
local_output_path=self._tmpdir.name,
cloud_output_path=self.cloud_output_path,
predictor_type=self._predictor_type,
resource_prefix=f"ag-cloud-{self.model_id}",
role=role,
)

Expand Down
8 changes: 4 additions & 4 deletions src/autogluon/cloud/predictor/cloud_predictor.py
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,7 @@ def fit(
If `custom_image_uri` is set, this argument will be ignored.
job_name: str, default = None
Name of the launched training job.
If None, CloudPredictor will create one with prefix ag-cloudpredictor
If None, CloudPredictor creates one with a predictor-specific prefix.
instance_type: str, default = 'ml.m5.2xlarge'
Instance type the predictor will be trained on with SageMaker.
instance_count: Union[int, str], default = "auto"
Expand Down Expand Up @@ -405,7 +405,7 @@ def deploy(
If None, will deploy the most recent trained predictor trained with `fit()`.
endpoint_name: str
The endpoint name to use for the deployment.
If None, CloudPredictor will create one with prefix `ag-cloudpredictor`
If None, CloudPredictor creates one with a predictor-specific prefix.
framework_version: str, default = `latest`
Inference container version of autogluon.
If `latest`, will use the latest available container version.
Expand Down Expand Up @@ -594,7 +594,7 @@ def predict(
If `custom_image_uri` is set, this argument will be ignored.
job_name: str, default = None
Name of the launched training job.
If None, CloudPredictor will create one with prefix ag-cloudpredictor.
If None, CloudPredictor creates one with a predictor-specific prefix.
instance_count: int, default = 1,
Number of instances used to do batch transform.
instance_type: str, default = 'ml.m5.2xlarge'
Expand Down Expand Up @@ -689,7 +689,7 @@ def predict_proba(
If `custom_image_uri` is set, this argument will be ignored.
job_name: str, default = None
Name of the launched training job.
If None, CloudPredictor will create one with prefix ag-cloudpredictor.
If None, CloudPredictor creates one with a predictor-specific prefix.
instance_count: int, default = 1,
Number of instances used to do batch transform.
instance_type: str, default = 'ml.m5.2xlarge'
Expand Down
10 changes: 10 additions & 0 deletions src/autogluon/cloud/predictor/multimodal_cloud_predictor.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import logging
import warnings

from ..backend.constant import MULTIMODL_SAGEMAKER, SAGEMAKER
from .cloud_predictor import CloudPredictor
Expand All @@ -16,6 +17,15 @@ class MultiModalCloudPredictor(CloudPredictor):
predictor_file_name = "MultiModalCloudPredictor.pkl"
backend_map = {SAGEMAKER: MULTIMODL_SAGEMAKER}

def __init__(self, *args, **kwargs) -> None:
warnings.warn(
"AutoGluon Multimodal is on a deprecation path. "
"MultiModalCloudPredictor will be removed in autogluon.cloud v0.7.0.",
FutureWarning,
stacklevel=2,
)
super().__init__(*args, **kwargs)

@property
def predictor_type(self) -> str:
"""
Expand Down
4 changes: 2 additions & 2 deletions src/autogluon/cloud/predictor/tabular_cloud_predictor.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ def fit_predict(
Training container version of autogluon. If `latest`, will use the latest available container version.
If `custom_image_uri` is set, this argument will be ignored.
job_name: str, default = None
Name of the launched training job. If None, CloudPredictor will create one with prefix ag-cloudpredictor.
Name of the launched training job. If None, CloudPredictor creates one with prefix ``ag-cloud-tabular``.
instance_type: str, default = 'ml.m5.2xlarge'
Instance type the predictor will be trained on with SageMaker.
instance_count: int, default = 1
Expand Down Expand Up @@ -169,7 +169,7 @@ def fit_predict_proba(
framework_version: str, default = `latest`
Training container version of autogluon. If `custom_image_uri` is set, this argument is ignored.
job_name: str, default = None
Name of the launched training job. If None, CloudPredictor will create one with prefix ag-cloudpredictor.
Name of the launched training job. If None, CloudPredictor creates one with prefix ``ag-cloud-tabular``.
instance_type: str, default = 'ml.m5.2xlarge'
Instance type the predictor will be trained on with SageMaker.
instance_count: int, default = 1
Expand Down
6 changes: 3 additions & 3 deletions src/autogluon/cloud/predictor/timeseries_cloud_predictor.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ def fit(
If `custom_image_uri` is set, this argument will be ignored.
job_name: str, default = None
Name of the launched training job.
If None, CloudPredictor will create one with prefix ag-cloudpredictor
If None, CloudPredictor creates one with prefix ``ag-cloud-timeseries``.
instance_type: str, default = 'ml.m5.2xlarge'
Instance type the predictor will be trained on with SageMaker.
instance_count: int, default = 1
Expand Down Expand Up @@ -263,7 +263,7 @@ def predict(
If `custom_image_uri` is set, this argument will be ignored.
job_name: str, default = None
Name of the launched training job.
If None, CloudPredictor will create one with prefix ag-cloudpredictor.
If None, CloudPredictor creates one with prefix ``ag-cloud-timeseries``.
instance_count: int, default = 1,
Number of instances used to do batch transform.
instance_type: str, default = 'ml.m5.2xlarge'
Expand Down Expand Up @@ -382,7 +382,7 @@ def fit_predict(
Training container version of autogluon. If `latest`, will use the latest available container version.
If `custom_image_uri` is set, this argument will be ignored.
job_name: str, default = None
Name of the launched training job. If None, CloudPredictor will create one with prefix ag-cloudpredictor.
Name of the launched training job. If None, CloudPredictor creates one with prefix ``ag-cloud-timeseries``.
instance_type: str, default = 'ml.m5.2xlarge'
Instance type the predictor will be trained on with SageMaker.
instance_count: int, default = 1
Expand Down
2 changes: 0 additions & 2 deletions src/autogluon/cloud/utils/constants.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
VALID_ACCEPT = ["application/x-parquet", "text/csv", "application/json"]

CLOUD_RESOURCE_PREFIX = "ag-cloudpredictor"

LOCAL_MODE = "local"
LOCAL_MODE_GPU = "local_gpu"
MODEL_ARTIFACT_NAME = "model.tar.gz"
Expand Down
Loading