Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
292 changes: 163 additions & 129 deletions .github/conda-lock/conda-linux-64-py3.11.lock

Large diffs are not rendered by default.

294 changes: 164 additions & 130 deletions .github/conda-lock/conda-linux-64-py3.12.lock

Large diffs are not rendered by default.

292 changes: 163 additions & 129 deletions .github/conda-lock/conda-linux-64-py3.13.lock

Large diffs are not rendered by default.

292 changes: 163 additions & 129 deletions .github/conda-lock/conda-linux-64-py3.14.lock

Large diffs are not rendered by default.

88 changes: 60 additions & 28 deletions .github/conda-lock/conda-osx-arm64-py3.11.lock

Large diffs are not rendered by default.

88 changes: 60 additions & 28 deletions .github/conda-lock/conda-osx-arm64-py3.12.lock

Large diffs are not rendered by default.

88 changes: 60 additions & 28 deletions .github/conda-lock/conda-osx-arm64-py3.13.lock

Large diffs are not rendered by default.

88 changes: 60 additions & 28 deletions .github/conda-lock/conda-osx-arm64-py3.14.lock

Large diffs are not rendered by default.

5 changes: 2 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ To maintain high code quality and consistency, we use several automated tools. W
| Target | Description | When to run |
| :--- | :--- | :--- |
| `make format` | Automatically format all code using `black` | Frequently during development |
| `make lint` | Run all linters (`black`, `flake8`, `mypy`, `pylint`) in parallel | Before every commit |
| `make lint` | Run all linters (`black`, `flake8`, `ty`, `pylint`) in parallel | Before every commit |
| `make test` | Run fast unit tests in parallel | Regularly during development |
| `make unit-tests` | Run all unit tests with 100% per-component coverage check | Before pushing |
| `make test-ci` | Run the full test suite exactly as the CI system does | Final check before pushing |
Expand All @@ -56,6 +56,5 @@ works, and how the CI pipeline is structured, see

- Use `black` for formatting.
- Follow PEP 8 guidelines (enforced by `flake8`).
- Use type hints wherever possible (checked by `mypy`).
- Use type hints wherever possible (checked by `type checker`).
- Ensure `pylint` passes without warnings in the relevant packages.

4 changes: 2 additions & 2 deletions CONTRIBUTING_ADVANCED.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ graph TD
%% Lint dependencies
lint --> lint-black["lint-black"]
lint --> lint-flake8["lint-flake8"]
lint --> lint-mypy["lint-mypy"]
lint --> lint-ty["lint-ty"]
lint --> lint-pylint["lint-pylint"]

%% Documentation dependencies
Expand Down Expand Up @@ -95,7 +95,7 @@ graph TD
%% Lint parallelism
lint --> lint-black["lint-black"]:::sequential
lint --> lint-flake8["lint-flake8"]:::sequential
lint --> lint-mypy["lint-mypy"]:::sequential
lint --> lint-ty["lint-ty"]:::sequential
lint --> lint-pylint["lint-pylint<br/>(parallel)"]:::parallel

lint-pylint --> pylint-fc["pylint-firecrown"]:::sequential
Expand Down
15 changes: 7 additions & 8 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ SHELL := /bin/bash
.PHONY: help format lint typecheck test test-coverage test-example test-integration test-slow \
test-all clean clean-docs clean-coverage docs tutorials api-docs docs-build \
lint-black lint-flake8 lint-pylint lint-pylint-firecrown lint-pylint-plugins \
lint-pylint-tests lint-pylint-examples lint-mypy pre-commit install all-checks \
lint-pylint-tests lint-pylint-examples lint-ty pre-commit install all-checks \
test-updatable test-utils test-parameters test-modeling-tools \
test-models-cluster test-models-two-point unit-tests test-ci test-all-coverage \
unit-tests-pre unit-tests-post unit-tests-core docs-generate-symbol-map \
Expand Down Expand Up @@ -146,7 +146,7 @@ conda-lock-check: ## Verify generated lockfiles are up to date

##@ Linting

lint: lint-black lint-flake8 lint-mypy lint-pylint ## Run all linting tools
lint: lint-black lint-flake8 lint-ty lint-pylint ## Run all linting tools
@echo "✅ All linters passed!"

lint-black: ## Check code formatting with black
Expand All @@ -159,10 +159,10 @@ lint-flake8: ## Run flake8 linter
@flake8 $(FIRECROWN_PKG_DIR)/ $(EXAMPLES_DIR)/ $(TESTS_DIR)/ || (echo "❌ flake8 failed" && exit 1)
@echo "✅ flake8 passed"

lint-mypy: ## Run mypy type checker
@echo "Running mypy..."
@mypy -p $(FIRECROWN_PKG_DIR) -p $(EXAMPLES_DIR) -p $(TESTS_DIR) || (echo "❌ mypy failed" && exit 1)
@echo "✅ mypy passed"
lint-ty: ## Run ty type checker
@echo "Running ty..."
@ty check || (echo "❌ ty failed" && exit 1)
@echo "✅ ty passed"

lint-pylint: lint-pylint-firecrown lint-pylint-plugins lint-pylint-tests lint-pylint-examples ## Run all pylint checks
@echo "✅ All pylint checks passed!"
Expand All @@ -187,7 +187,7 @@ lint-pylint-examples: ## Run pylint on examples
@pylint --rcfile $(EXAMPLES_DIR)/pylintrc $(EXAMPLES_DIR) || (echo "❌ pylint failed for examples" && exit 1)
@echo "✅ pylint passed for examples"

typecheck: lint-mypy ## Alias for mypy type checking
typecheck: lint-ty ## Alias for ty type checking

##@ Testing

Expand Down Expand Up @@ -239,7 +239,6 @@ test-models-two-point unit-tests-core test-slow test-example test-integration: |
unit-tests-post: test-updatable test-utils test-parameters test-modeling-tools test-models-cluster test-models-two-point
@echo "Combining coverage data..."
@COVERAGE_FILE=$(UNIT_COVERAGE_COMBINED) coverage combine $(UNIT_COVERAGE_FILES)
@COVERAGE_FILE=$(UNIT_COVERAGE_COMBINED) coverage report

test-updatable: ## Run tests for firecrown.updatable module with coverage
@COVERAGE_FILE=$(UNIT_COVERAGE_UPDATABLE) $(PYTEST) tests/test_updatable.py \
Expand Down
2 changes: 1 addition & 1 deletion Making_a_new_release.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ This is the procedure we use for create a new release of Firecrown.
1. Create a local branch in your clone of the repo to be used for the PR that will be started later. Name it for the new release, e.g.: prep-v1.10. Note that this is not the branch that would be used for bug fixes. A bug fix branch would only be created at the time it becomes necessary.
1. Make sure you have updated the `environment.yml` file to release, or force, any versioning required.
1. Update your local conda environment using `conda env update --file environment.yml --prune -q --json`.
1. Make sure any new directories are being processed by black, flake8, mypy, and pylint.
1. Make sure any new directories are being processed by black, flake8, ty, and pylint.
1. Update the version in `firecrown/version.py` (this is the ONLY file you need to edit manually).
1. Update the tutorial version metadata by running:

Expand Down
5 changes: 2 additions & 3 deletions docs/contrib.rst
Original file line number Diff line number Diff line change
Expand Up @@ -63,12 +63,12 @@ Type checking
-------------

We are using type-hinting in (most of) the code, to help ensure correct use of the framework.
We are using :bash:`mypy` to verify the code is conforming to these type hints.
We are using :bash:`ty` to verify the code is conforming to these type hints.
Please run:

.. code:: bash

mypy -p firecrown -p examples -p tests
ty check

and fix any errors reported before pushing commits to the GitHub repository.

Expand All @@ -84,4 +84,3 @@ To run the same set of tests that will be run by the CI system, use:

Please note that when the CI system runs the tests, it will also ensure that all modified or newly-added code is actually tested.
For a PR to be reviewed, it must pass this requirement.

4 changes: 2 additions & 2 deletions docs/developer_installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -121,8 +121,8 @@ All of these are used as part of the CI system as part of the checking of all pu
pylint firecrown
pylint --rcfile tests/pylintrc tests

# We are using type hints and mypy to help catch type-related errors.
mypy -p firecrown -p examples -p tests
# We are using type hints and type checker to help catch type-related errors.
ty check

Keeping your conda environment up-to-date
=========================================
Expand Down
3 changes: 2 additions & 1 deletion environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ dependencies:
- charset-normalizer
- clmm
- compilers
- conda-lock
- cosmosis >= 3.0
- cosmosis-build-standard-library
- coverage
Expand All @@ -26,7 +27,6 @@ dependencies:
- linkchecker
- matplotlib-base
- more-itertools
- mypy
- numcosmo >= 0.21.1
- numpy >= 2.0, < 2.4
- pip >= 20.1 # pip is needed as dependency
Expand Down Expand Up @@ -57,6 +57,7 @@ dependencies:
- sphinx-rtd-theme
- sphinxcontrib-apidoc
- tabulate
- ty == 0.0.34
- typer
- types-pyyaml
- types-requests
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,11 @@
from numcosmo_py import Ncm
import sacc

CLUSTER_COUNTS = getattr(sacc.standard_types, "cluster_counts", "cluster_counts")
CLUSTER_MEAN_LOG_MASS = getattr(
sacc.standard_types, "cluster_mean_log_mass", "cluster_mean_log_mass"
)


def generate_SDSSCL_sacc_file() -> None:
"""
Expand Down Expand Up @@ -58,9 +63,9 @@ def generate_SDSSCL_sacc_file() -> None:
bin_richness_labels.append(bin_richness_label)

# pylint: disable-next=no-member
cluster_count = sacc.standard_types.cluster_counts
cluster_count = CLUSTER_COUNTS
# pylint: disable-next=no-member
cluster_mean_log_mass = sacc.standard_types.cluster_mean_log_mass
cluster_mean_log_mass = CLUSTER_MEAN_LOG_MASS

counts_and_edges = zip(
cluster_counts.flatten(), itertools.product(bin_z_labels, bin_richness_labels)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,12 @@
import sacc
import pyccl as ccl

CLUSTER_COUNTS = getattr(sacc.standard_types, "cluster_counts", "cluster_counts")
CLUSTER_MEAN_LOG_MASS = getattr(
sacc.standard_types, "cluster_mean_log_mass", "cluster_mean_log_mass"
)
CLUSTER_SHEAR = getattr(sacc.standard_types, "cluster_shear", "cluster_shear")

os.environ["CLMM_MODELING_BACKEND"] = "ccl"
# pylint: disable=C0413
import clmm # noqa: E402
Expand Down Expand Up @@ -322,11 +328,11 @@ def generate_sacc_file() -> None:
bin_radius_labels.append(bin_radius_label)

# pylint: disable-next=no-member
cluster_count = sacc.standard_types.cluster_counts
cluster_count = CLUSTER_COUNTS
# pylint: disable-next=no-member
cluster_mean_log_mass = sacc.standard_types.cluster_mean_log_mass
cluster_mean_log_mass = CLUSTER_MEAN_LOG_MASS
# pylint: disable-next=no-member
cluster_mean_DeltaSigma = sacc.standard_types.cluster_shear
cluster_mean_DeltaSigma = CLUSTER_SHEAR

counts_and_edges = zip(
cluster_counts.flatten(), itertools.product(bin_z_labels, bin_richness_labels)
Expand Down
13 changes: 10 additions & 3 deletions firecrown/app/examples/_cosmic_shear.py
Original file line number Diff line number Diff line change
Expand Up @@ -293,7 +293,10 @@ def _show_covariance_config(self, table: Table) -> None:
table.add_row("Format", "diagonal")

def _create_tracers(
self, sacc_data: sacc.Sacc, cosmo: pyccl.Cosmology, z_range: np.ndarray
self,
sacc_data: sacc.Sacc,
cosmo: pyccl.Cosmology,
z_range: np.ndarray,
) -> tuple[np.ndarray, list[pyccl.WeakLensingTracer]]:
"""Create tomographic redshift bins and weak lensing tracers.

Expand Down Expand Up @@ -375,17 +378,21 @@ def _add_covariance_matrix(
covariance = np.diag(cov_diag)
sacc_data.add_covariance(covariance)

def generate_factory(self, output_path: Path, _sacc: Path) -> Path:
# pylint: disable=redefined-outer-name
def generate_factory(self, output_path: Path, sacc: Path) -> Path:
"""Copy cosmic shear factory template.

:param output_path: Output directory
:param _sacc: SACC file path (unused)
:param sacc: SACC file path (unused)
:return: Path to factory file
"""
del sacc
output_file = output_path / f"{self.prefix}_factory.py"
copy_template(_cosmic_shear_template, output_file)
return output_file

# pylint: enable=redefined-outer-name

def get_build_parameters(self, sacc_path: Path) -> NamedParameters:
"""Return SACC file path for likelihood construction."""
return NamedParameters(
Expand Down
7 changes: 5 additions & 2 deletions firecrown/app/examples/_sn_srd.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,13 +62,16 @@ def generate_sacc(self, output_path: Path) -> Path:
)
return sacc_full_file

def generate_factory(self, output_path: Path, _sacc: Path) -> Path:
def generate_factory(
self, output_path: Path, sacc: Path
) -> Path: # pylint: disable=redefined-outer-name
"""Copy supernova factory template.

:param output_path: Output directory
:param _sacc: SACC file path (unused)
:param sacc: SACC file path (unused)
:return: Path to factory file
"""
del sacc
output_file = output_path / f"{self.prefix}_factory.py"
copy_template(_sn_srd_template, output_file)
return output_file
Expand Down
2 changes: 1 addition & 1 deletion firecrown/connector/numcosmo/helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ def get_amplitude_parameters(
A_s: float | None = None
sigma8: float | None = None

# mypy verifies that the match statement below is exhaustive
# Type checker verifies that the match statement below is exhaustive
match amplitude_parameter:
case fac.PoweSpecAmplitudeParameter.SIGMA8:
if p_ml is None:
Expand Down
16 changes: 12 additions & 4 deletions firecrown/connector/numcosmo/numcosmo.py
Original file line number Diff line number Diff line change
Expand Up @@ -585,17 +585,24 @@ def do_prepare( # pylint: disable-msg=arguments-differ
raise_on_unused=self.likelihood.raise_on_unused_parameter,
)

def do_m2lnL_val(self, _) -> float: # pylint: disable-msg=arguments-differ
# pylint: disable=arguments-differ
def do_m2lnL_val(
self,
mset: Ncm.MSet,
) -> float:
"""Implements the virtual method `m2lnL`.

This method should calculate the value of the likelihood for
the model set `mset`.

:param _: unused, but required by interface
:param mset: model set (unused)
"""
del mset
loglike = self.likelihood.compute_loglike_for_sampling(self.tools)
return -2.0 * loglike

# pylint: enable=arguments-differ


class NumCosmoGaussCov(Ncm.DataGaussCov):
"""NumCosmoGaussCov is a subclass of Ncm.DataGaussCov.
Expand Down Expand Up @@ -896,15 +903,16 @@ def do_prepare( # pylint: disable-msg=arguments-differ
self.tools.prepare()

# pylint: disable-next=arguments-differ
def do_mean_func(self, _, vp) -> None:
def do_mean_func(self, mset: Ncm.MSet, vp: Ncm.Vector) -> None:
"""Implements the virtual `Ncm.DataGaussCov` method `mean_func`.

This method should compute the theoretical mean for the gaussian
distribution.

:param _: unused, but required by interface
:param mset: model set (unused)
:param vp: the vector to set
"""
del mset
theory_vector = self.likelihood.compute_theory_vector(self.tools)
vp.set_array(theory_vector)

Expand Down
6 changes: 3 additions & 3 deletions firecrown/fctools/test_ci_local.sh
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,8 @@ black --check firecrown tests examples
echo " - Running flake8..."
flake8 firecrown examples tests

echo " - Running mypy..."
mypy -p firecrown -p examples -p tests
echo " - Running ty..."
ty check

echo " - Running pylint on firecrown..."
pylint firecrown
Expand All @@ -84,4 +84,4 @@ python -m pytest -vv -s --example -m example tests/example

echo ""
echo "✅ Local CI test completed successfully!"
echo "This simulates the macOS Python 3.13 matrix job (non-coverage version)"
echo "This simulates the macOS Python 3.13 matrix job (non-coverage version)"
15 changes: 10 additions & 5 deletions firecrown/likelihood/_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -382,7 +382,7 @@ def __init__(self, parameter_prefix: None | str = None):
self.computed_theory_vector = False
self.theory_vector: None | TheoryVector = None

def read(self, _: sacc.Sacc) -> None:
def read(self, sacc_data: sacc.Sacc) -> None:
"""Read the data for this statistic and mark it as ready for use.

Derived classes that override this function should make sure to call the
Expand All @@ -394,8 +394,9 @@ def read(self, _: sacc.Sacc) -> None:

as the last thing they do.

:param _: currently unused, but required by the interface.
:param sacc_data: currently unused, but required by the interface.
"""
del sacc_data
assert len(self.get_data_vector()) > 0
self.ready = True

Expand Down Expand Up @@ -566,12 +567,13 @@ def get_data_vector(self) -> DataVector:
assert self.data_vector is not None
return self.data_vector

def _compute_theory_vector(self, _: ModelingTools) -> TheoryVector:
def _compute_theory_vector(self, tools: ModelingTools) -> TheoryVector:
"""Return a fixed theory vector.

:param _: unused, but required by the interface
:param tools: unused, but required by the interface
:return: A fixed theory vector
"""
del tools
return TheoryVector.from_list([self.mean] * self.count)


Expand Down Expand Up @@ -1023,8 +1025,11 @@ def __init__(self, sacc_tracer: str, active: bool = True) -> None:
else:
self._transform = dndz_shift_and_stretch_passive

def apply(self, _: ModelingTools, tracer_arg: _SourceGalaxyArgsT):
def apply(
self, tools: ModelingTools, tracer_arg: _SourceGalaxyArgsT
) -> _SourceGalaxyArgsT:
"""Apply a shift & stretch to the photo-z distribution of a source."""
del tools
new_z, new_dndz = self._transform(
tracer_arg.z, tracer_arg.dndz, self.delta_z, self.sigma_z
)
Expand Down
2 changes: 1 addition & 1 deletion firecrown/likelihood/_gaussfamily.py
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ def wrapper_repeat(*args: P.args, **kwargs: P.kwargs) -> T:
succeeds.
"""
# The odd use of args[0] instead of self seems to be the only way
# to have both the Python runtime and mypy agree on what is being
# to have both the Python runtime and type checker agree on what is being
# passed to the method, and to allow access to the attribute
# 'state'. Recall that the syntax:
# o.foo()
Expand Down
Loading
Loading