diff --git a/Makefile b/Makefile index c00130c9c0..e33d43ed1b 100644 --- a/Makefile +++ b/Makefile @@ -16,7 +16,7 @@ YCONTRIB_NXDL_TARGETS = $(patsubst %.yaml,%.nxdl.xml,$(subst /nyaml/,/, $(wildca YAPPDEF_NXDL_TARGETS = $(patsubst %.yaml,%.nxdl.xml,$(subst /nyaml/,/, $(wildcard $(APPDEF_DIR)/nyaml/*.yaml))) -.PHONY: help install style autoformat test clean prepare html pdf impatient-guide all local nxdl nyaml +.PHONY: help install style autoformat test docdiff clean prepare html pdf impatient-guide all local nxdl nyaml help :: @echo "" @@ -27,6 +27,8 @@ help :: @echo "make style Check python coding style." @echo "make autoformat Format all files to the coding style conventions." @echo "make test Run NXDL syntax and documentation tests." + @echo "make docdiff Diff the generated documentation between two git" + @echo " references. Default: REFS=\"HEAD~1 HEAD\"." @echo "make clean Remove all build files." @echo "make prepare (Re)create all build files." @echo "make html Build HTML version of manual. Requires prepare first." @@ -58,6 +60,10 @@ autoformat :: test :: $(PYTHON) -m pytest dev_tools +# for example: make docdiff REFS="HEAD~3 ." NXCLASS="-c NXstress" +docdiff :: + $(PYTHON) -m dev_tools docdiff $(NXCLASS) $(REFS) + clean :: $(RM) -rf $(BUILD_DIR) $(RM) -rf $(BASE_CLASS_DIR)/$(NYAML_SUBDIR) diff --git a/applications/NXapm.nxdl.xml b/applications/NXapm.nxdl.xml index 033ede220b..54238a6adc 100644 --- a/applications/NXapm.nxdl.xml +++ b/applications/NXapm.nxdl.xml @@ -69,6 +69,21 @@ Number of mass resolution values. + + Number of pixels along the i direction of a two-dimensional image. + + + Number of pixels along the j direction of a two-dimensional image. + + + Number of points along the x direction of a three-dimensional grid. + + + Number of points along the y direction of a three-dimensional grid. + + + Number of points along the z direction of a three-dimensional grid. + Application definition for real or simulated atom probe and field-ion microscopy experiments. diff --git a/applications/NXellipsometry.nxdl.xml b/applications/NXellipsometry.nxdl.xml index 88b091d515..487ffd2eb1 100644 --- a/applications/NXellipsometry.nxdl.xml +++ b/applications/NXellipsometry.nxdl.xml @@ -51,6 +51,12 @@ Number of angles of incidence of the incident beam. + + Number of observables of the measured data (2nd dimension of the measured_data array), as described by 'data_type'. + + + Number of sensors, one for each varied parameter. + This is the application definition describing ellipsometry experiments. diff --git a/applications/NXstress.nxdl.xml b/applications/NXstress.nxdl.xml index a6aa29403a..0b67124868 100644 --- a/applications/NXstress.nxdl.xml +++ b/applications/NXstress.nxdl.xml @@ -43,12 +43,8 @@ Converted diffractogram X units (could be the same as *x_Unit*). - - number of temperatures - - - number of values in applied stress field - + + number of scan points (only present in scanning measurements) @@ -59,6 +55,20 @@ number of detector pixels in the second (faster) direction + + + Concepts that this application definition does not add anything to, but that + are documented here because they are relevant for stress and strain analysis. + + + + + + + + + + Application definition for stress and strain analysis of crystalline material defined by the `EASI-STRESS consortium <https://easi-stress.eu>`_. @@ -106,9 +116,6 @@ - - Extended title for the entry. - Unique identifier for the experiment as defined by the facility (e.g. DOI, proposal id, ...). At ILL, this could be, for example, ``exp_1-02-286``, ``exp_INDU-229``, or ``exp_INTER-569``. @@ -196,10 +203,8 @@ Information about the person who performed the experiment. - - - Role of user responsible for this entry. Suggested roes are, for example, ``local contact``, ``beamline_scientist``, ``post_doc``,… + Role of user responsible for this entry. Suggested roles are, for example, ``local contact``, ``beamline_scientist``, ``post_doc``,… @@ -258,9 +263,6 @@ Zero or more of these groups describe the detectors used in the experiment. - - name/manufacturer/model/etc. information - Description of type such as \ :sup:`3`\ He gas cylinder, \ :sup:`3`\ He PSD, scintillator, fission chamber, proportion counter, ion chamber, CCD, pixel, image plate, CMOS, … @@ -309,20 +311,6 @@ - - Detector dead time - - - - - - - - Elapsed actual counting time - - - - The axis on which the detector position depends may be stored @@ -442,34 +430,6 @@ Descriptive name of sample - - - The chemical formula specified using CIF conventions. - Abbreviated version of CIF standard: - - * Only recognized element symbols may be used. - * Each element symbol is followed by a 'count' number. A count of '1' may be omitted. - * A space or parenthesis must separate each cluster of (element symbol + count). - * Where a group of elements is enclosed in parentheses, the multiplier for the - group must follow the closing parentheses. That is, all element and group - multipliers are assumed to be printed as subscripted numbers. - * Unless the elements are ordered in a manner that corresponds to their chemical - structure, the order of the elements within any group or moiety depends on - whether or not carbon is present. - * If carbon is present, the order should be: - - - C, then H, then the other elements in alphabetical order of their symbol. - - If carbon is not present, the elements are listed purely in alphabetic order of their symbol. - - * This is the *Hill* system used by Chemical Abstracts. - - - - Sample temperature. This could be a scanned variable - - - - Applied external stress field @@ -587,8 +547,6 @@ Information about the person who performed the data reduction. - - Role of user responsible for this entry. Suggested roles are, for example, ``local contact``, ``beamline_scientist``, ``post_doc``,… diff --git a/applications/NXtomophase.nxdl.xml b/applications/NXtomophase.nxdl.xml index 5aa673c93d..267a8b71f9 100644 --- a/applications/NXtomophase.nxdl.xml +++ b/applications/NXtomophase.nxdl.xml @@ -159,7 +159,7 @@ fluctuations in the beam between frames. - + diff --git a/base_classes/NXbeam.nxdl.xml b/base_classes/NXbeam.nxdl.xml index e14afec573..78e5f2243f 100644 --- a/base_classes/NXbeam.nxdl.xml +++ b/base_classes/NXbeam.nxdl.xml @@ -39,6 +39,12 @@ Number of moments representing beam divergence (x, y, xy, etc.) + + Number of delay points in the FROG trace. + + + Number of frequency points in the FROG trace. + Properties of the neutron or X-ray beam at a given location. diff --git a/base_classes/NXdetector.nxdl.xml b/base_classes/NXdetector.nxdl.xml index 9ba868a1e9..c523822891 100644 --- a/base_classes/NXdetector.nxdl.xml +++ b/base_classes/NXdetector.nxdl.xml @@ -44,6 +44,9 @@ number of detector pixels in the second (faster) direction number of detector pixels in the third (if necessary, fastest) direction number of bins in the time-of-flight histogram + + number of entries in the count-rate correction lookup table + @@ -731,7 +734,7 @@ In cases where the data is of type :ref:`NXlog` this can also be an NXlog. - + diff --git a/base_classes/NXdetector_group.nxdl.xml b/base_classes/NXdetector_group.nxdl.xml index 0856325645..d795df0777 100644 --- a/base_classes/NXdetector_group.nxdl.xml +++ b/base_classes/NXdetector_group.nxdl.xml @@ -26,6 +26,11 @@ xsi:schemaLocation="http://definition.nexusformat.org/nxdl/3.1 ../nxdl.xsd" name="NXdetector_group" type="group" extends="NXobject"> + + + number of detectors and groups of detectors + + Logical grouping of detectors. When used, describes a group of detectors. diff --git a/base_classes/NXevent_data.nxdl.xml b/base_classes/NXevent_data.nxdl.xml index 196945b3b3..cd123b4138 100644 --- a/base_classes/NXevent_data.nxdl.xml +++ b/base_classes/NXevent_data.nxdl.xml @@ -26,6 +26,17 @@ xsi:schemaLocation="http://definition.nexusformat.org/nxdl/3.1 ../nxdl.xsd" name="NXevent_data" type="group" extends="NXobject"> + + + number of events + + + number of pulses + + + number of detector ends that are read out for each event + + NXevent_data is a special group for storing data from neutron detectors in event mode. In this mode, the detector electronics diff --git a/base_classes/NXfilter.nxdl.xml b/base_classes/NXfilter.nxdl.xml index 396d59c647..dcd090baaf 100644 --- a/base_classes/NXfilter.nxdl.xml +++ b/base_classes/NXfilter.nxdl.xml @@ -30,6 +30,14 @@ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://definition.nexusformat.org/nxdl/3.1 ../nxdl.xsd" > + + + number of compositions of the filter material + + + number of reflecting surfaces + + For band pass beam filters. diff --git a/base_classes/NXoff_geometry.nxdl.xml b/base_classes/NXoff_geometry.nxdl.xml index 49827450d8..a63a543c18 100644 --- a/base_classes/NXoff_geometry.nxdl.xml +++ b/base_classes/NXoff_geometry.nxdl.xml @@ -37,6 +37,9 @@ detecting volumes + + length of the winding order list, which is the total number of vertices of all faces + diff --git a/base_classes/NXorientation.nxdl.xml b/base_classes/NXorientation.nxdl.xml index 5544314ed3..c261b890d9 100644 --- a/base_classes/NXorientation.nxdl.xml +++ b/base_classes/NXorientation.nxdl.xml @@ -27,6 +27,11 @@ xsi:schemaLocation="http://definition.nexusformat.org/nxdl/3.1 ../nxdl.xsd" name="NXorientation" type="group" extends="NXobject"> + + + number of objects whose orientation is described + + legacy class - recommend to use :ref:`NXtransformations` now diff --git a/base_classes/NXpositioner.nxdl.xml b/base_classes/NXpositioner.nxdl.xml index 82d69c63a2..5b39a26110 100644 --- a/base_classes/NXpositioner.nxdl.xml +++ b/base_classes/NXpositioner.nxdl.xml @@ -30,6 +30,11 @@ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://definition.nexusformat.org/nxdl/3.1 ../nxdl.xsd" > + + + number of positioner values, which is more than one when the positioner is scanned + + A generic positioner such as a motor or piezo-electric transducer. diff --git a/base_classes/NXsensor.nxdl.xml b/base_classes/NXsensor.nxdl.xml index 63b5e348ef..cb16cac50f 100644 --- a/base_classes/NXsensor.nxdl.xml +++ b/base_classes/NXsensor.nxdl.xml @@ -26,6 +26,11 @@ xsi:schemaLocation="http://definition.nexusformat.org/nxdl/3.1 ../nxdl.xsd" name="NXsensor" type="group" extends="NXcomponent"> + + + number of sensor values, which is more than one when the value is a vector + + A sensor used to monitor an external condition diff --git a/base_classes/NXshape.nxdl.xml b/base_classes/NXshape.nxdl.xml index 03272c9222..03d4e38a6e 100644 --- a/base_classes/NXshape.nxdl.xml +++ b/base_classes/NXshape.nxdl.xml @@ -26,6 +26,14 @@ xsi:schemaLocation="http://definition.nexusformat.org/nxdl/3.1 ../nxdl.xsd" name="NXshape" type="group" extends="NXobject"> + + + number of objects whose shape is described + + + number of parameters describing the shape + + legacy class - (used by :ref:`NXgeometry`) - the shape and size of a component. diff --git a/base_classes/NXtranslation.nxdl.xml b/base_classes/NXtranslation.nxdl.xml index 5e47b1d647..745edb4f3d 100644 --- a/base_classes/NXtranslation.nxdl.xml +++ b/base_classes/NXtranslation.nxdl.xml @@ -26,6 +26,11 @@ xsi:schemaLocation="http://definition.nexusformat.org/nxdl/3.1 ../nxdl.xsd" name="NXtranslation" type="group" extends="NXobject"> + + + number of objects whose position is described + + legacy class - (used by :ref:`NXgeometry`) - general spatial location of a component. diff --git a/contributed_definitions/NXapm_compositionspace_results.nxdl.xml b/contributed_definitions/NXapm_compositionspace_results.nxdl.xml index f3fdfbcf88..e80888e55d 100644 --- a/contributed_definitions/NXapm_compositionspace_results.nxdl.xml +++ b/contributed_definitions/NXapm_compositionspace_results.nxdl.xml @@ -41,6 +41,12 @@ Total number of ions in the reconstructed dataset. + + Number of values along the axis of the respective one-dimensional plot. + + + Number of labels returned by the clustering. + Application definition for results of the CompositionSpace tool used in atom probe. diff --git a/contributed_definitions/NXapm_paraprobe_clusterer_config.nxdl.xml b/contributed_definitions/NXapm_paraprobe_clusterer_config.nxdl.xml index ac32ea4b9d..a1574d8545 100644 --- a/contributed_definitions/NXapm_paraprobe_clusterer_config.nxdl.xml +++ b/contributed_definitions/NXapm_paraprobe_clusterer_config.nxdl.xml @@ -42,7 +42,30 @@ Number of different iontypes to distinguish during clustering. + + Number of eps respectively min_cluster_size parameter values. + + + Number of min_pts respectively min_samples parameter values. + + + Number of cluster_selection_epsilon parameter values. + + + Number of alpha parameter values. + + + + The tomographic reconstruction that this analysis is based on is specified + with concepts of :ref:`NXapm_paraprobe_tool_config`. + + + + + + + Application definition for a configuration file of the paraprobe-clusterer tool. @@ -63,13 +86,6 @@ a binary file that is formatted like a POS file but cluster labels written out using floating point numbers. - - - - - - - File with the results of the cluster analyses that was computed with IVAS / AP suite diff --git a/contributed_definitions/NXapm_paraprobe_clusterer_results.nxdl.xml b/contributed_definitions/NXapm_paraprobe_clusterer_results.nxdl.xml index 761480c5a6..778bfc7c99 100644 --- a/contributed_definitions/NXapm_paraprobe_clusterer_results.nxdl.xml +++ b/contributed_definitions/NXapm_paraprobe_clusterer_results.nxdl.xml @@ -36,6 +36,12 @@ Number of clusters found. + + Number of targets, i.e. ions of the reconstruction that were considered during clustering. + + + Number of labelled targets returned by the clustering backend, which is not necessarily the number of targets. + Application definition for a results file of the paraprobe-clusterer tool. diff --git a/contributed_definitions/NXapm_paraprobe_nanochem_results.nxdl.xml b/contributed_definitions/NXapm_paraprobe_nanochem_results.nxdl.xml index c483f2e6ec..283b498ce8 100644 --- a/contributed_definitions/NXapm_paraprobe_nanochem_results.nxdl.xml +++ b/contributed_definitions/NXapm_paraprobe_nanochem_results.nxdl.xml @@ -83,6 +83,27 @@ The cardinality/total number of triangles in the triangle soup.--> The total number of ROIs placed in a oned_profile task. + + Number of entries of the respective array, e.g. the number of vertices or the number of features. + + + Number of entries of the respective array, e.g. the number of faces or triangles. + + + Number of entries of the respective array, e.g. the number of triangle normals or edges. + + + The total number of ions inside a polyhedron. + + + The total number of faces of a polyhedron. + + + The total number of vertices of a polyhedron. + + + The total number of voxels of the scalar field that is rendered via XDMF. + Application definition for a results file of the paraprobe-nanochem tool. diff --git a/contributed_definitions/NXapm_paraprobe_surfacer_results.nxdl.xml b/contributed_definitions/NXapm_paraprobe_surfacer_results.nxdl.xml index a707c4596a..a413c8df3a 100644 --- a/contributed_definitions/NXapm_paraprobe_surfacer_results.nxdl.xml +++ b/contributed_definitions/NXapm_paraprobe_surfacer_results.nxdl.xml @@ -51,6 +51,9 @@ The total number of XDMF values to represent all faces of tetrahedra via XDMF. + + The total number of vertices of the tetrahedra. + Application definition for a results file of the paraprobe-surfacer tool. diff --git a/contributed_definitions/NXcontainer.nxdl.xml b/contributed_definitions/NXcontainer.nxdl.xml index e73c4f8064..211531e34c 100644 --- a/contributed_definitions/NXcontainer.nxdl.xml +++ b/contributed_definitions/NXcontainer.nxdl.xml @@ -29,6 +29,11 @@ type="group" extends="NXcomponent" > + + + number of compositions of the material the container is made from + + State of a container holding the sample under investigation. diff --git a/contributed_definitions/NXmicrostructure.nxdl.xml b/contributed_definitions/NXmicrostructure.nxdl.xml index 6c003a023c..80d5a453c7 100644 --- a/contributed_definitions/NXmicrostructure.nxdl.xml +++ b/contributed_definitions/NXmicrostructure.nxdl.xml @@ -114,6 +114,9 @@ The number of line defects.--> configuration/dimensionality + + Number of junctions of the respective type. + Base class to describe a microstructure, its structural aspects, associated descriptors, properties. diff --git a/contributed_definitions/NXmicrostructure_score_results.nxdl.xml b/contributed_definitions/NXmicrostructure_score_results.nxdl.xml index 71b1aded91..ba679e93ce 100644 --- a/contributed_definitions/NXmicrostructure_score_results.nxdl.xml +++ b/contributed_definitions/NXmicrostructure_score_results.nxdl.xml @@ -69,6 +69,15 @@ inspect comments behind NXmicrostructure--> Number of grains in the computer simulation + + Number of grid points along the x direction. + + + Number of grid points along the y direction. + + + Number of grid points along the z direction. + Application definition for storing results of the SCORE cellular automata model. diff --git a/contributed_definitions/NXmicrostructure_slip_system.nxdl.xml b/contributed_definitions/NXmicrostructure_slip_system.nxdl.xml index 7384fbd117..2c3ab93ce8 100644 --- a/contributed_definitions/NXmicrostructure_slip_system.nxdl.xml +++ b/contributed_definitions/NXmicrostructure_slip_system.nxdl.xml @@ -36,6 +36,9 @@ Number of indices used for reporting Miller (3) or Miller-Bravais indices (4). + + Number of Miller indices describing a crystallographic plane respectively direction. + Base class for describing a set of crystallographic slip systems. diff --git a/contributed_definitions/NXoptical_polarizer.nxdl.xml b/contributed_definitions/NXoptical_polarizer.nxdl.xml index d12a6ea077..a1a31798af 100644 --- a/contributed_definitions/NXoptical_polarizer.nxdl.xml +++ b/contributed_definitions/NXoptical_polarizer.nxdl.xml @@ -38,6 +38,15 @@ A draft of a new base class to describe an optical polarizer the polarizer is given. + + Number of objects the device is made up of. + + + Number of parameters describing the shape of the objects. + + + Length of the spectrum array of the coating. + An optical polarizer. diff --git a/contributed_definitions/NXsnsevent.nxdl.xml b/contributed_definitions/NXsnsevent.nxdl.xml index 934ada4985..7447bd3623 100644 --- a/contributed_definitions/NXsnsevent.nxdl.xml +++ b/contributed_definitions/NXsnsevent.nxdl.xml @@ -8,6 +8,29 @@ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://definition.nexusformat.org/nxdl/3.1 ../nxdl.xsd" > + + + number of values in the log + + + number of values in the log + + + number of detector pixels in the x direction + + + number of detector pixels in the y direction + + + number of pulses + + + number of events + + + number of time channels of the time-of-flight histogram + + This is a definition for event data from Spallation Neutron Source (SNS) at ORNL. diff --git a/contributed_definitions/NXsnshisto.nxdl.xml b/contributed_definitions/NXsnshisto.nxdl.xml index 2e94b7c71e..f2aae59918 100644 --- a/contributed_definitions/NXsnshisto.nxdl.xml +++ b/contributed_definitions/NXsnshisto.nxdl.xml @@ -8,6 +8,26 @@ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://definition.nexusformat.org/nxdl/3.1 ../nxdl.xsd" > + + + number of values in the log + + + number of values in the log + + + number of detector pixels in the x direction + + + number of detector pixels in the y direction + + + number of time-of-flight bins + + + number of time channels of the time-of-flight histogram + + This is a definition for histogram data from Spallation Neutron Source (SNS) at ORNL. diff --git a/contributed_definitions/NXspm_scan_pattern.nxdl.xml b/contributed_definitions/NXspm_scan_pattern.nxdl.xml index 5a594e4903..0c3c9b8116 100644 --- a/contributed_definitions/NXspm_scan_pattern.nxdl.xml +++ b/contributed_definitions/NXspm_scan_pattern.nxdl.xml @@ -22,6 +22,14 @@ # For further information, see http://www.nexusformat.org --> + + + number of trajectory points + + + number of dimensions of the phase space in which the trajectory is defined + + Basic base class to define the pattern of a scan in a given scan region. diff --git a/contributed_definitions/NXxrd.nxdl.xml b/contributed_definitions/NXxrd.nxdl.xml index 203475211e..43b3642c77 100644 --- a/contributed_definitions/NXxrd.nxdl.xml +++ b/contributed_definitions/NXxrd.nxdl.xml @@ -25,6 +25,11 @@ ! : additions ? : could or should be modified?--> + + + number of detector channels of the diffractogram + + NXxrd on top of NXmonopd diff --git a/contributed_definitions/NXxrd_pan.nxdl.xml b/contributed_definitions/NXxrd_pan.nxdl.xml index eb3923bdba..85c76b5c8b 100644 --- a/contributed_definitions/NXxrd_pan.nxdl.xml +++ b/contributed_definitions/NXxrd_pan.nxdl.xml @@ -22,6 +22,11 @@ # For further information, see http://www.nexusformat.org --> + + + number of detector channels of the diffractogram + + NXxrd_pan is a specialization of NXxrd with extra properties for the PANalytical XRD data format. diff --git a/dev_tools/__main__.py b/dev_tools/__main__.py index f2a1bc8b06..8a05b86ee6 100644 --- a/dev_tools/__main__.py +++ b/dev_tools/__main__.py @@ -2,6 +2,7 @@ import sys from .apps import dir_app +from .apps import docdiff_app from .apps import impatient_app from .apps import manual_app from .apps import nxclass_app @@ -35,6 +36,12 @@ def main(argv=None): test_app.nxtest_args(nxtest_parser) dir_app.dir_args(nxtest_parser) + docdiff_parser = subparsers.add_parser( + "docdiff", help="Diff the generated documentation of two git references" + ) + docdiff_app.docdiff_args(docdiff_parser) + dir_app.dir_args(docdiff_parser) + if argv is None: argv = sys.argv args = parser.parse_args(argv[1:]) @@ -44,6 +51,7 @@ def main(argv=None): "manual": manual_app.manual_exec, "impatient": impatient_app.impatient_exec, "nxtest": test_app.nxtest_exec, + "docdiff": docdiff_app.docdiff_exec, }.get(args.command) if app_exec is None: diff --git a/dev_tools/apps/docdiff_app.py b/dev_tools/apps/docdiff_app.py new file mode 100644 index 0000000000..ba5b0826a5 --- /dev/null +++ b/dev_tools/apps/docdiff_app.py @@ -0,0 +1,333 @@ +"""Compare the generated documentation of two git references.""" + +import contextlib +import difflib +import fnmatch +import os +import shutil +import signal +import subprocess +import sys +import traceback +from pathlib import Path +from typing import Iterator +from typing import List +from typing import Optional +from typing import Sequence + +from ..globals import directories + +# reference name for the working tree, which includes uncommitted changes +WORKING_TREE = "." + +# generated files that are compared (globs relative to the sphinx source directory) +GENERATED_PATTERNS = ( + "classes/*/*.rst", + "nxdl_desc.rst", + "units.table", + "types.table", + "_static/nxdl_vocabulary.txt", +) + +_ANSI = { + "+": "\033[32m", + "-": "\033[31m", + "@": "\033[36m", +} +_ANSI_RESET = "\033[0m" + + +def docdiff_args(parser): + parser.add_argument( + "refs", + nargs="*", + metavar="REF", + help="Git references to compare, for example 'HEAD~2 HEAD' or a branch " + f"name. Use '{WORKING_TREE}' for the working tree, which includes " + f"uncommitted changes. Default: 'HEAD~1 HEAD'", + ) + parser.add_argument( + "-c", + "--nxclass", + action="append", + metavar="NAME", + help="Only compare this NeXus class (for example 'NXstress'). Repeatable. " + "Default: all classes", + ) + parser.add_argument( + "--context", + type=int, + default=3, + metavar="N", + help="Number of context lines in the diff. Default: 3", + ) + parser.add_argument( + "--color", + choices=["auto", "always", "never"], + default="auto", + help="Colorize the diff. Default: auto", + ) + parser.add_argument( + "--output", + type=str, + metavar="FILE", + help="Write the diff to this file instead of stdout", + ) + parser.add_argument( + "--keep", + action="store_true", + help="Keep the generated documentation of both references", + ) + parser.add_argument( + "--verbose", + action="store_true", + help="Print the output of the documentation generation", + ) + parser.add_argument( + "--exit-code", + action="store_true", + help="Exit with 1 when the documentation differs", + ) + + +def docdiff_exec(args) -> int: + refs = args.refs or ["HEAD~1", "HEAD"] + if len(refs) != 2: + print(f"Expected two git references, got {len(refs)}", file=sys.stderr) + return 1 + repo = Path(directories.get_source_root()) + root = Path(directories.get_build_root()) / "docdiff" + + generated = list() + try: + for name, ref in zip(("old", "new"), refs): + print(f"generate the documentation of '{ref}' ...", file=sys.stderr) + generated.append( + _generate_docs( + repo, ref, root / name, args.nxclass, verbose=args.verbose + ) + ) + diff_lines = list( + _diff_directories( + *generated, *refs, GENERATED_PATTERNS, context=args.context + ) + ) + except KeyboardInterrupt: + # nothing is left behind when interrupted, also not with --keep + with contextlib.suppress(KeyboardInterrupt): + _remove_generated(repo, root) + print("\ndocdiff: interrupted", file=sys.stderr) + return 1 + except Exception as e: + with contextlib.suppress(KeyboardInterrupt): + _remove_generated(repo, root) + if args.verbose: + traceback.print_exc() + print(f"docdiff: {e}", file=sys.stderr) + return 1 + if not args.keep: + _remove_generated(repo, root) + + nfiles = sum(line.startswith("--- ") for line in diff_lines) + if args.output: + with open(args.output, "w") as fh: + fh.writelines(diff_lines) + print(f"{nfiles} file(s) differ, diff written to {args.output}") + else: + color = args.color == "always" or (args.color == "auto" and sys.stdout.isatty()) + sys.stdout.writelines(_colorize(diff_lines) if color else diff_lines) + print(f"\n{nfiles} file(s) differ", file=sys.stderr) + if args.keep: + print( + f"generated documentation in {root}\n" + "remove the git worktrees with 'git worktree prune' after deleting it", + file=sys.stderr, + ) + + return 1 if diff_lines and args.exit_code else 0 + + +def _generate_docs( + repo: Path, + ref: str, + dest: Path, + nxclasses: Optional[Sequence[str]], + verbose: bool = False, +) -> Path: + """Generate the NeXus class documentation of a git reference. Returns the + directory with the generated files. + """ + _remove_generated(repo, dest) + if ref == WORKING_TREE: + source_root = repo + else: + source_root = dest / "source" + _git(repo, "worktree", "add", "--detach", str(source_root), ref) + build_root = dest / "build" + + if nxclasses: + commands = [["nxclass", name, "--prepare"] for name in nxclasses] + else: + commands = [["manual", "--prepare"]] + # the documentation is generated by the dev_tools of the reference itself + env = dict(os.environ) + pythonpath = [str(source_root)] + if env.get("PYTHONPATH"): + pythonpath.append(env["PYTHONPATH"]) + env["PYTHONPATH"] = os.pathsep.join(pythonpath) + env.pop("NEXUS_DEF_PATH", None) + for command in commands: + _run_generator( + [sys.executable, "-m", "dev_tools"] + + command + + ["--build-root", str(build_root)], + ref, + source_root, + env, + verbose, + ) + return build_root / "manual" / "source" + + +def _run_generator( + command: Sequence[str], ref: str, cwd: Path, env: dict, verbose: bool +) -> None: + """Run a documentation generator in its own process group, so that a CTRL-C + in the terminal is handled here and the process is never left behind. + """ + with subprocess.Popen( + command, + cwd=cwd, + env=env, + stdout=None if verbose else subprocess.DEVNULL, + stderr=None if verbose else subprocess.PIPE, + text=not verbose, + start_new_session=True, + ) as process: + try: + _, stderr = process.communicate() + except BaseException: + _kill(process) + raise + if process.returncode: + raise RuntimeError( + f"generating the documentation of '{ref}' failed " + f"(exit code {process.returncode})\n{(stderr or '').strip()}" + ) + + +def _kill(process: subprocess.Popen) -> None: + """Kill a process and everything it started.""" + with contextlib.suppress(BaseException): + os.killpg(os.getpgid(process.pid), signal.SIGKILL) + with contextlib.suppress(BaseException): + process.kill() + + +def _remove_generated(repo: Path, root: Path) -> None: + """Remove the generated documentation and unregister the git worktrees. + Interrupting this leaves nothing behind, it cannot be interrupted. + """ + with _delayed_interrupt(): + for source_root in [root / "source"] + [ + root / name / "source" for name in ("old", "new") + ]: + if not source_root.is_dir(): + continue + with contextlib.suppress(BaseException): + _git( + repo, "worktree", "remove", "--force", str(source_root), detach=True + ) + shutil.rmtree(root, ignore_errors=True) + with contextlib.suppress(BaseException): + _git(repo, "worktree", "prune", detach=True) + + +@contextlib.contextmanager +def _delayed_interrupt(): + """Handle SIGINT (CTRL-C) when leaving the context instead of inside it.""" + interrupted = False + + def handler(*_): + nonlocal interrupted + interrupted = True + + try: + original = signal.signal(signal.SIGINT, handler) + except ValueError: + yield # not the main thread + return + try: + yield + finally: + signal.signal(signal.SIGINT, original) + if interrupted: + raise KeyboardInterrupt + + +def _git(repo: Path, *args: str, detach: bool = False) -> str: + """`detach` shields the git process from a CTRL-C in the terminal.""" + result = subprocess.run( + ["git"] + list(args), + cwd=repo, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + start_new_session=detach, + ) + if result.returncode in (-signal.SIGINT, -signal.SIGTERM): + raise KeyboardInterrupt + if result.returncode: + raise RuntimeError(f"'git {' '.join(args)}' failed: {result.stderr.strip()}") + return result.stdout + + +def _diff_directories( + old_root: Path, + new_root: Path, + old_ref: str, + new_ref: str, + patterns: Sequence[str], + context: int = 3, +) -> Iterator[str]: + """Unified diff of the generated files of two references.""" + for relpath in _iter_generated(old_root, new_root, patterns): + old_lines = _read(old_root / relpath) + new_lines = _read(new_root / relpath) + if old_lines == new_lines: + continue + yield from difflib.unified_diff( + old_lines, + new_lines, + fromfile=f"{old_ref}:{relpath}", + tofile=f"{new_ref}:{relpath}", + n=context, + ) + + +def _iter_generated( + old_root: Path, new_root: Path, patterns: Sequence[str] +) -> Iterator[Path]: + """Files generated for either reference, in a reproducible order.""" + relpaths = set() + for root in (old_root, new_root): + for path in root.rglob("*"): + if not path.is_file(): + continue + relpath = path.relative_to(root) + if any(fnmatch.fnmatch(str(relpath), pattern) for pattern in patterns): + relpaths.add(relpath) + yield from sorted(relpaths) + + +def _read(path: Path) -> List[str]: + if not path.is_file(): + return list() + with open(path, "r") as fh: + return list(fh) + + +def _colorize(lines: Sequence[str]) -> Iterator[str]: + for line in lines: + color = _ANSI.get(line[:1]) if not line.startswith(("+++", "---")) else None + yield f"{color}{line}{_ANSI_RESET}" if color else line diff --git a/dev_tools/docs/anchor_list.py b/dev_tools/docs/anchor_list.py index 020eb89da5..48de17267e 100644 --- a/dev_tools/docs/anchor_list.py +++ b/dev_tools/docs/anchor_list.py @@ -118,7 +118,7 @@ def write(self): datetime=utcnow, title="NeXus NXDL vocabulary.", subtitle="Anchors for all NeXus fields, groups, " - "attributes, and links.", + "attributes, links, and symbols.", version=get_nxdl_version(), ), terms=self._registry, diff --git a/dev_tools/docs/nxdl.py b/dev_tools/docs/nxdl.py index ea80cca733..1ac14beee9 100644 --- a/dev_tools/docs/nxdl.py +++ b/dev_tools/docs/nxdl.py @@ -3,8 +3,11 @@ from collections import OrderedDict from html import parser as HTMLParser from pathlib import Path +from typing import Dict +from typing import Iterator from typing import List from typing import Optional +from typing import Tuple import lxml @@ -22,6 +25,65 @@ MIN_COLLAPSE_HINT_LINE_LENGTH = 20 MAX_COLLAPSE_HINT_LINE_LENGTH = 80 +# how a concept that is shown in the documentation of a class that does not +# define it is marked +REUSED_MARKER = "*(reused)*" + +# which children are shown along with a reused concept +REUSE_CHILDREN_MODES = ("none", "direct", "all") + +# maximal number of levels below a reused concept with `children="all"` +MAX_REUSE_DEPTH = 5 + +# maximal number of reused concepts in one class +MAX_REUSED_CONCEPTS = 200 + +# symbol names in a dimension size, which can be an expression like "nP+1" +SYMBOL_PATTERN = re.compile(r"[A-Za-z_][A-Za-z_0-9]*") + +# characters that may precede or follow inline markup in reStructuredText +RST_INLINE_PREFIX = " \t-:/'\"<([{" +RST_INLINE_SUFFIX = " \t-.,:;!?\\/'\")]}>" + +_BACKTICK_RUN = re.compile(r"`+") +_ASTERISK_RUN = re.compile(r"\*+") + + +class ReusedConcept: + """A group, field or attribute that a class does not define itself but that is + shown in its documentation. + + A class takes over everything defined by the class it ``extends`` and by the base + classes of the groups it uses, without repeating any of it. Only what a class + defines itself is shown in its documentation. The ``reused_concepts`` element of + NXDL points at concepts that a class takes over unchanged, so that they are shown + as well. Reusing a concept changes nothing about the class itself. + """ + + def __init__( + self, + name: str, + is_attribute: bool, + parent: Optional["ReusedConcept"], + node: lxml.etree._Element, + defined_here: bool, + ) -> None: + self.name = name + self.is_attribute = is_attribute + self.parent = parent + self.node = node + self.defined_here = defined_here + parent_path = parent.path if parent is not None else "" + self.path = f"{parent_path}{'@' if is_attribute else '/'}{name}" + self.nxdl_path = f"{parent.nxdl_path if parent is not None else ''}/{name}" + self.listed = False + self.children_mode = "none" + self.children: Dict[str, "ReusedConcept"] = OrderedDict() + + @property + def element_type(self) -> str: + return xml_utils.get_local_name(self.node) + class NXClassDocGenerator: """Generate documentation in reStructuredText markup @@ -42,6 +104,14 @@ def _reset(self): self._anchor_registry = None self._listing_category = None self._use_application_defaults = None + self._nxclass_name = None + self._nxdl_file = None + self._root_element = None + self._reused_concepts = OrderedDict() + self._reused_index = dict() + self._reused_count = 0 + self._declared_symbols = set() + self._used_symbols = dict() def __call__( self, nxdl_file: PathLike, anchor_registry: Optional[AnchorRegistry] = None @@ -54,8 +124,8 @@ def __call__( try: try: self._parse_nxdl_file(nxdl_file) - except Exception: - raise NXDLParseError(nxdl_file) + except Exception as e: + raise NXDLParseError(f"{nxdl_file}: {e}") from e finally: self._reset() return self._rst_lines @@ -83,6 +153,9 @@ def _parse_nxdl_file(self, nxdl_file: Path): self._listing_category = self._CATEGORY_TO_LISTING[category] self._use_application_defaults = category == "application" self._contribution = nxdl_file.parent.name == "contributed_definitions" + self._nxclass_name = nxclass_name + self._nxdl_file = nxdl_file + self._root_element = root # print ReST comments and section header source = os.path.relpath(nxdl_file, get_nxdl_root()) @@ -154,11 +227,25 @@ def _parse_nxdl_file(self, nxdl_file: Path): else: self._print_doc_enum("", ns, node_list[0]) for node in node_list[0].xpath("nx:symbol", namespaces=ns): - doc = self._get_doc_line(ns, node) - self._print(f" **{node.get('name')}**", end="") + name = node.get("name") + reused_from = node.get("reused_from") + self._declared_symbols.add(name) + if reused_from: + doc = self._reused_symbol_doc(ns, node, name, reused_from) + suffix = f" :ref:`⤆ ` {REUSED_MARKER}" + else: + doc = self._get_doc_line(ns, node) + suffix = "" + self._print( + f" {self._hyperlink_target(f'/{nxclass_name}', name, 'symbol')}" + ) + self._print(f" **{name}**", end="") if doc: self._print(f": {doc}", end="") - self._print("\n") + self._print(f"{suffix}\n") + + # concepts of other classes that are shown in the documentation of this one + self._parse_reused_concepts(ns, root) # print group references self._print("**Groups cited**:") @@ -168,6 +255,12 @@ def _parse_nxdl_file(self, nxdl_file: Path): g = node.get("type") if g.startswith("NX") and g not in groups: groups.append(g) + for concept in self._iter_reused_concepts(): + if concept.element_type != "group": + continue + g = concept.node.get("type") + if g.startswith("NX") and g not in groups: + groups.append(g) if len(groups) == 0: self._print(" none\n") else: @@ -183,6 +276,7 @@ def _parse_nxdl_file(self, nxdl_file: Path): # print full tree self._print("**Structure**:\n") + self._print_reuse_legend() for subnode in root.xpath("nx:attribute", namespaces=ns): optional = self._get_required_or_optional_text(subnode) self._print_attribute( @@ -192,6 +286,8 @@ def _parse_nxdl_file(self, nxdl_file: Path): ns, root, nxclass_name, self._INDENTATION_UNIT, parent_path ) + self._check_symbols() + self._print_anchor_list() # print NXDL source location @@ -217,7 +313,7 @@ def _print_anchor_list(self): self._print("-----------------\n") self._print( "List of hypertext anchors for all groups, fields,\n" - "attributes, and links defined in this class.\n\n" + "attributes, links, and symbols defined in this class.\n\n" ) def sorter(key): @@ -344,31 +440,37 @@ def _get_doc_line(self, ns, node): return self._handle_multiline_docstring(blocks) return blocks[0].replace("\n", " ") - def _get_minOccurs(self, node): + def _get_minOccurs(self, node, use_application_defaults=None): """ get the value for the ``minOccurs`` attribute :param obj node: instance of lxml.etree._Element + :param use_application_defaults: defaults of the class defining the node :returns str: value of the attribute (or its default) """ # TODO: can we improve on the default by examining nxdl.xsd? - minOccurs_default = str(int(self._use_application_defaults)) + if use_application_defaults is None: + use_application_defaults = self._use_application_defaults + minOccurs_default = str(int(use_application_defaults)) minOccurs = node.get("minOccurs", minOccurs_default) return minOccurs - def _get_required_or_optional_text(self, node): + def _get_required_or_optional_text(self, node, use_application_defaults=None): """ make clear if a reported item is required or optional :param obj node: instance of lxml.etree._Element + :param use_application_defaults: defaults of the class defining the node :returns: formatted text """ + if use_application_defaults is None: + use_application_defaults = self._use_application_defaults nxdl_element_type = nxdl_utils.get_nxdl_element_type(node) if nxdl_element_type in ("field", "group", "choice"): - optional_default = not self._use_application_defaults + optional_default = not use_application_defaults optional = node.get("optional", optional_default) in (True, "true", "1", 1) recommended = node.get("recommended", None) in (True, "true", "1", 1) - minOccurs = self._get_minOccurs(node) + minOccurs = self._get_minOccurs(node, use_application_defaults) if recommended: optional_text = "(recommended) " elif minOccurs in ("0", 0) or optional: @@ -380,7 +482,7 @@ def _get_required_or_optional_text(self, node): # TODO: add a remark to the log optional_text = f"(``minOccurs={str(minOccurs)}``) " elif nxdl_element_type in ("attribute",): - optional_default = not self._use_application_defaults + optional_default = not use_application_defaults optional = node.get("optional", optional_default) in (True, "true", "1", 1) recommended = node.get("recommended", None) in (True, "true", "1", 1) optional_text = {True: "(optional) ", False: "(required) "}[optional] @@ -449,6 +551,9 @@ def _analyze_dimensions(self, ns, parent) -> str: # Dimension symbol dim = subnode.get("value") # integer or symbol from the table + if dim: + self._register_dimension_symbols(dim, parent) + dim = self._link_dimension_symbols(dim) if not dim: ref = subnode.get("ref") if ref: @@ -576,9 +681,42 @@ def long_doc(self, ns, node, left_margin): for single_line in lines: if len(single_line) > 2 and single_line[0] != "." and not fnd: fnd = True - line = single_line[:max_characters] + line = self._truncate_rst_line(single_line, max_characters) return (length, line, blocks) + @staticmethod + def _has_unbalanced_runs(text: str, pattern: "re.Pattern[str]") -> bool: + """Whether text has an odd number of matching delimiter runs, e.g. a `role` or ``literal`` + cut off before its closing backticks.""" + stack: List[str] = [] + for run in pattern.findall(text): + if stack and stack[-1] == run: + stack.pop() + else: + stack.append(run) + return bool(stack) + + @staticmethod + def _truncate_rst_line(text: str, max_length: int) -> str: + """Truncate text to at most max_length characters without leaving unterminated RST inline + markup (a role/link cut mid-``target``) or a dangling implicit hyperlink reference (word_). + """ + candidate = text[:max_length] + while candidate: + stripped = candidate.rstrip() + if ( + not stripped.endswith("_") + and not NXClassDocGenerator._has_unbalanced_runs( + stripped, _BACKTICK_RUN + ) + and not NXClassDocGenerator._has_unbalanced_runs( + stripped, _ASTERISK_RUN + ) + ): + return candidate + candidate = candidate.rsplit(" ", 1)[0] if " " in candidate else "" + return candidate + def _print_doc_enum(self, indent, ns, node, required=False): collapse_indent = indent node_list = node.xpath("nx:enumeration", namespaces=ns) @@ -594,7 +732,9 @@ def _print_doc_enum(self, indent, ns, node, required=False): collapse_indent + self._INDENTATION_UNIT, ns, node_list[0] ) - def _print_attribute(self, ns, kind, node, optional, indent, parent_path): + def _print_attribute( + self, ns, kind, node, optional, indent, parent_path, reused=False + ): name = node.get("name") formatted_name = nxdl_utils.get_rst_formatted_name(node) index_name = name @@ -602,8 +742,12 @@ def _print_attribute(self, ns, kind, node, optional, indent, parent_path): f"{indent}" f"{self._hyperlink_target(parent_path, name, 'attribute')}" ) self._print(f"{indent}.. index:: {index_name} ({kind} attribute)\n") + if reused: + reference = f"{self._reused_ref(node, 'attribute')} {REUSED_MARKER}" + else: + reference = self.get_first_parent_ref(f"{parent_path}/{name}", "attribute") self._print( - f"{indent}{formatted_name}: {optional}{self._format_type(node)}{self._format_units(node)} {self.get_first_parent_ref(f'{parent_path}/{name}', 'attribute')}\n" + f"{indent}{formatted_name}: {optional}{self._format_type(node)}{self._format_units(node)} {reference}\n" ) self._print_if_deprecated(ns, node, indent + self._INDENTATION_UNIT) self._print_doc_enum(indent, ns, node) @@ -624,9 +768,27 @@ def _print_full_tree(self, ns, parent, name, indent, parent_path): :param indent: to keep track of indentation level :param parent_path: NX class path of parent nodes """ + # Reused concepts are shown where the class would define them: attributes + # right after the ones of the class itself, fields before the first group + # and groups last. + reused = self._reused_index.get(parent_path, ()) + reused_attributes = [c for c in reused if c.element_type == "attribute"] + reused_fields = [c for c in reused if c.element_type in ("field", "link")] + reused_groups = [c for c in reused if c.element_type == "group"] + for concept in reused_attributes: + self._print_reused_concept(ns, concept, indent) + fields_printed = False + # Process children in document order to preserve XML ordering. for node in parent.xpath("nx:field|nx:group|nx:choice|nx:link", namespaces=ns): nxdl_element_type = nxdl_utils.get_nxdl_element_type(node) + if not fields_printed and xml_utils.get_local_name(node) in ( + "group", + "choice", + ): + fields_printed = True + for concept in reused_fields: + self._print_reused_concept(ns, concept, indent) if nxdl_element_type == "field": name = node.get("name") @@ -663,6 +825,10 @@ def _print_full_tree(self, ns, parent, name, indent, parent_path): parent_path + "/" + name, ) + self._print_reused_concepts( + ns, indent + self._INDENTATION_UNIT, parent_path + "/" + name + ) + elif nxdl_element_type == "group": name = node.get("name", "") formatted_name = nxdl_utils.get_rst_formatted_name(node) @@ -763,6 +929,389 @@ def _print_full_tree(self, ns, parent, name, indent, parent_path): else: raise ValueError(f"Unknown node type: {nxdl_element_type}") + if not fields_printed: + for concept in reused_fields: + self._print_reused_concept(ns, concept, indent) + for concept in reused_groups: + self._print_reused_concept(ns, concept, indent) + + def _parse_reused_concepts(self, ns, root) -> None: + """Parse the ``reused_concepts`` element: concepts of other classes that are + shown in the documentation of this class. + """ + node_list = root.xpath("nx:reused_concepts", namespaces=ns) + if not node_list: + return + if len(node_list) > 1: + raise ValueError(f"Invalid reused_concepts list in {self._nxclass_name}") + + listed = list() + for node in node_list[0].xpath("nx:reuse", namespaces=ns): + path = node.get("path") + children_mode = node.get("children", "none") + if children_mode not in REUSE_CHILDREN_MODES: + raise ValueError( + f"'{path}': children='{children_mode}' is not one of " + f"{', '.join(REUSE_CHILDREN_MODES)}" + ) + concept = None + concepts = self._reused_concepts + for name, is_attribute in self._split_reuse_path(path): + key = f"@{name}" if is_attribute else name + child = concepts.get(key) + if child is None: + child = self._create_reused_concept(concept, name, is_attribute) + concepts[key] = child + concept = child + concepts = concept.children + if concept.listed: + raise ValueError(f"'{path}' is reused more than once") + if concept.defined_here: + raise ValueError( + f"'{path}' is defined by {self._nxclass_name} itself: remove the " + "definition or remove it from the 'reused_concepts' list" + ) + concept.listed = True + concept.children_mode = children_mode + listed.append(concept) + + for concept in listed: + self._expand_reused_concept(ns, concept, concept.children_mode, 1) + self._index_reused_concepts(self._reused_concepts, f"/{self._nxclass_name}") + + @staticmethod + def _split_reuse_path(path: str) -> List[Tuple[str, bool]]: + """Split a ``reuse`` path into ``(name, is_attribute)`` per level.""" + if not path or not path.startswith("/"): + raise ValueError(f"reuse path '{path}' must start with '/'") + segments = list() + for part in path[1:].split("/"): + names = part.split("@") + if len(names) > 2 or not all(names): + raise ValueError(f"'{path}' is not a valid reuse path") + segments.append((names[0], False)) + if len(names) == 2: + segments.append((names[1], True)) + return segments + + def _create_reused_concept( + self, parent: Optional[ReusedConcept], name: str, is_attribute: bool + ) -> ReusedConcept: + nxdl_path = f"{parent.nxdl_path if parent is not None else ''}/{name}" + elist = nxdl_utils.get_inherited_nodes(nxdl_path, None, self._root_element)[2] + if not elist: + raise ValueError(f"'{nxdl_path}' does not exist in {self._nxclass_name}") + node = elist[0] + element_type = xml_utils.get_local_name(node) + if element_type == "choice": + raise ValueError( + f"'{nxdl_path}' is a choice, which cannot be highlighted in the " + "documentation" + ) + if is_attribute != (element_type == "attribute"): + raise ValueError( + f"'{nxdl_path}' is a {element_type}: an attribute is separated from " + "its parent with '@', anything else with '/'" + ) + defined_name = nxdl_utils.get_node_name(node) + if defined_name != name: + raise ValueError( + f"'{nxdl_path}' must be spelled '{defined_name}' as in the class that defines it" + ) + # nodes of the class being documented have an empty 'nxdlbase' + defined_here = not node.get("nxdlbase") + if not defined_here: + self._check_not_renamed(parent, node, name, nxdl_path) + return ReusedConcept(name, is_attribute, parent, node, defined_here) + + def _check_not_renamed( + self, parent: Optional[ReusedConcept], node, name: str, nxdl_path: str + ) -> None: + """A concept that this class redefines under another name, such as a group + with a flexible name that the class names itself, is not the same concept.""" + if parent is None: + parent_node = self._root_element + elif parent.defined_here: + parent_node = parent.node + else: + # this class does not define the parent, so it cannot redefine its children + return + own_node, _ = nxdl_utils.get_best_child( + parent_node, + None, + name, + nxdl_utils.get_nx_class(node), + nxdl_utils.get_nxdl_element_type(node), + ) + if own_node is None: + return + own_name = nxdl_utils.get_node_name(own_node) + if own_name != name: + raise ValueError( + f"'{nxdl_path}' is redefined by {self._nxclass_name} as " + f"'{own_name}': reuse that concept instead" + ) + + def _expand_reused_concept( + self, ns, concept: ReusedConcept, children_mode: str, level: int + ) -> None: + """Add the children of a reused concept that are shown as well.""" + if children_mode == "none" or concept.element_type not in ("group", "field"): + return + if level > MAX_REUSE_DEPTH: + raise ValueError( + f"'{concept.nxdl_path}' shows more than {MAX_REUSE_DEPTH} levels of " + "children: list the concepts to be shown instead of using " + "children='all'" + ) + child_mode = "all" if children_mode == "all" else "none" + for name, node in self._reused_children_nodes(ns, concept).items(): + if name in concept.children: + continue + child = ReusedConcept(name, False, concept, node, not node.get("nxdlbase")) + if child.defined_here or self._reused_cycle(child): + # documented where it is defined or already documented above + continue + child.listed = True + child.children_mode = child_mode + concept.children[name] = child + self._reused_count += 1 + if self._reused_count > MAX_REUSED_CONCEPTS: + raise ValueError( + f"more than {MAX_REUSED_CONCEPTS} concepts are reused: list the " + "concepts to be shown instead of using children='all'" + ) + self._expand_reused_concept(ns, child, child_mode, level + 1) + + def _reused_children_nodes(self, ns, concept: ReusedConcept) -> Dict: + """The groups, fields and links of a reused concept, including those of the + class it is typed as but excluding those that every group takes over from + NXobject.""" + elist = nxdl_utils.get_inherited_nodes( + concept.nxdl_path, None, self._root_element + )[2] + nodes = OrderedDict() + for elem in elist: + if elem.get("name") == "NXobject": + # every group has these, showing them everywhere is noise + continue + for child in elem.xpath("nx:field|nx:group|nx:link", namespaces=ns): + name = nxdl_utils.get_node_name(child) + if name not in nodes: + nodes[name] = nxdl_utils.set_nxdlpath(child, elem) + return nodes + + @staticmethod + def _reused_cycle(concept: ReusedConcept) -> bool: + """A concept or its group type is already reused by one of its own ancestors + (like NXsample in NXsample).""" + + def source(concept): + return concept.node.get("nxdlbase"), concept.node.get("nxdlpath") + + key = source(concept) + nxclass_name = ( + concept.node.get("type") if concept.element_type == "group" else None + ) + parent = concept.parent + while parent is not None: + if source(parent) == key: + return True + if nxclass_name is not None and parent.node.get("type") == nxclass_name: + return True + parent = parent.parent + return False + + def _index_reused_concepts(self, concepts, doc_parent_path: str) -> None: + """Index the concepts by the path at which they are documented. The children + of a concept that this class defines are documented below that definition.""" + for concept in concepts.values(): + if concept.defined_here: + self._index_reused_concepts( + concept.children, f"{doc_parent_path}/{concept.name}" + ) + else: + self._reused_index.setdefault(doc_parent_path, list()).append(concept) + + def _iter_reused_concepts(self, concepts=None) -> Iterator[ReusedConcept]: + if concepts is None: + concepts = self._reused_concepts + for concept in concepts.values(): + if not concept.defined_here: + yield concept + yield from self._iter_reused_concepts(concept.children) + + def _print_reuse_legend(self) -> None: + if not self._reused_concepts: + return + # the same indentation as the structure tree, else the tree ends up + # inside the note + indent = self._INDENTATION_UNIT + self._print( + f"{indent}.. note:: The ⤆ link points at the class a concept comes from. " + f"Items marked {REUSED_MARKER} are not defined by this class at all: they " + "are used exactly as that class defines them. Items with a ⤆ link but no " + "marker are defined by this class, which may change what they mean.\n" + ) + + def _print_reused_concepts(self, ns, indent: str, parent_path: str) -> None: + for concept in self._reused_index.get(parent_path, ()): + self._print_reused_concept(ns, concept, indent) + + def _print_reused_concept(self, ns, concept: ReusedConcept, indent: str) -> None: + node = concept.node + element_type = concept.element_type + name = concept.name + tag = "field" if element_type == "link" else element_type + doc_parent_path = f"/{self._nxclass_name}" + ( + concept.parent.path if concept.parent is not None else "" + ) + # occurrences follow the defaults of the class that defines the concept + use_application_defaults = node.get("nxdlbase_class") == "application" + optional_text = self._get_required_or_optional_text( + node, use_application_defaults + ) + + if element_type == "attribute": + self._print_attribute( + ns, + concept.parent.element_type if concept.parent is not None else "file", + node, + optional_text, + indent, + doc_parent_path, + reused=True, + ) + return + + marker = f"{self._reused_ref(node, tag)} {REUSED_MARKER}" + formatted_name = nxdl_utils.get_rst_formatted_name(node) + self._print(f"{indent}{self._hyperlink_target(doc_parent_path, name, tag)}") + if element_type == "group": + typ = node.get("type", "untyped (this is an error; please report)") + if typ.startswith("NX"): + typ = f":ref:`{typ}`" + self._print(f"{indent}{formatted_name}: {optional_text}{typ} {marker}\n") + elif element_type == "link": + self._print( + f"{indent}{formatted_name}: " + ":ref:`link` " + f"(suggested target: ``{node.get('target')}``)" + f" {marker}\n" + ) + else: + self._print(f"{indent}.. index:: {name} (field)\n") + self._print( + f"{indent}{formatted_name}: " + f"{optional_text}" + f"{self._format_type(node)}" + f"{self._analyze_dimensions(ns, node)}" + f"{self._format_units(node)}" + f" {marker}" + "\n" + ) + + self._print_if_deprecated(ns, node, indent + self._INDENTATION_UNIT) + + if concept.listed: + self._print_doc_enum(indent, ns, node) + for subnode in node.xpath("nx:attribute", namespaces=ns): + if f"@{subnode.get('name')}" in concept.children: + continue + nxdl_utils.set_nxdlpath(subnode, node) + optional = self._get_required_or_optional_text( + subnode, use_application_defaults + ) + self._print_attribute( + ns, + element_type, + subnode, + optional, + indent + self._INDENTATION_UNIT, + f"/{self._nxclass_name}{concept.path}", + reused=True, + ) + + for child in concept.children.values(): + self._print_reused_concept(ns, child, indent + self._INDENTATION_UNIT) + + @staticmethod + def _reused_ref(node, tag: str) -> str: + """Reference to the concept in the class that defines it.""" + nxdlbase = node.get("nxdlbase") + nxdlpath = node.get("nxdlpath") + if not nxdlbase or not nxdlpath: + return "" + nxclass_name = Path(nxdlbase).name.split(".")[0] + if tag == "attribute": + pos = nxdlpath.rfind("/") + nxdlpath = f"{nxdlpath[:pos]}@{nxdlpath[pos + 1:]}" + return f":ref:`⤆ `" + + def _reused_symbol_doc(self, ns, node, name: str, nxclass_name: str) -> str: + """Documentation of a symbol that is taken from another class.""" + if node.xpath("nx:doc", namespaces=ns): + raise ValueError( + f"symbol '{name}' has a 'doc' element as well as " + f"reused_from='{nxclass_name}'" + ) + nxdl_file = nxdl_utils.find_definition_file(nxclass_name) + if nxdl_file is None: + raise ValueError( + f"symbol '{name}' is reused from unknown class '{nxclass_name}'" + ) + root = xml_utils.read_xml_file(nxdl_file) + for symbol in root.xpath("nx:symbols/nx:symbol", namespaces=ns): + if symbol.get("name") == name: + return self._get_doc_line(ns, symbol) + raise ValueError(f"'{nxclass_name}' does not define the symbol '{name}'") + + def _link_dimension_symbols(self, size: str) -> str: + """Turn the symbols in the size of a dimension into links to the symbol + table of this class. + """ + parts = [] + end = 0 + for match in SYMBOL_PATTERN.finditer(size): + name = match.group() + if name not in self._declared_symbols: + continue + before = size[end : match.start()] + parts.append(before) + if before and before[-1] not in RST_INLINE_PREFIX: + parts.append("\\ ") + parts.append(f":ref:`{name} `") + end = match.end() + if end < len(size) and size[end] not in RST_INLINE_SUFFIX: + parts.append("\\ ") + parts.append(size[end:]) + return "".join(parts) + + def _register_dimension_symbols(self, size: str, node) -> None: + """Register the symbols used in the size of a dimension.""" + for symbol in SYMBOL_PATTERN.findall(size): + self._used_symbols.setdefault(symbol, nxdl_utils.get_node_name(node)) + + def _check_symbols(self) -> None: + """All symbols that appear in the documentation must be in the symbol table.""" + missing = { + symbol: used_by + for symbol, used_by in self._used_symbols.items() + if symbol not in self._declared_symbols + } + if not missing: + return + details = ", ".join( + f"'{symbol}' (dimension of '{used_by}')" + for symbol, used_by in sorted(missing.items()) + ) + raise ValueError( + f"{self._nxclass_name} documents symbols that are missing from its symbol " + f"table: {details}. Add them to the 'symbols' element, with a 'doc' element " + "or with a 'reused_from' attribute naming the class that documents the " + "symbol." + ) + def _print(self, *args, end="\n"): # TODO: change instances of \t to proper indentation self._rst_lines.append(" ".join(args) + end) diff --git a/dev_tools/tests/test_docdiff.py b/dev_tools/tests/test_docdiff.py new file mode 100644 index 0000000000..02261af214 --- /dev/null +++ b/dev_tools/tests/test_docdiff.py @@ -0,0 +1,100 @@ +"""Diffing the generated documentation of two git references.""" + +import subprocess +from pathlib import Path + +import pytest + +from ..__main__ import main +from ..apps import docdiff_app +from ..globals import directories + + +def _has_git_repository() -> bool: + try: + subprocess.run( + ["git", "rev-parse", "--git-dir"], + cwd=directories.get_source_root(), + check=True, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + ) + except (OSError, subprocess.CalledProcessError): + return False + return True + + +@pytest.fixture +def build_root(tmp_path): + original = directories.get_build_root() + yield tmp_path + directories.set_build_root(original) + + +@pytest.mark.skipif(not _has_git_repository(), reason="not a git repository") +def test_docdiff_same_reference(build_root, capsys): + exit_code = main( + [ + "dev_tools", + "docdiff", + "--nxclass", + "NXentry", + "--color", + "never", + "--exit-code", + "--build-root", + str(build_root), + "HEAD", + "HEAD", + ] + ) + captured = capsys.readouterr() + assert exit_code == 0, captured.err + assert not captured.out + assert "0 file(s) differ" in captured.err + assert not list(build_root.glob("docdiff/*")) + + +@pytest.mark.skipif(not _has_git_repository(), reason="not a git repository") +def test_docdiff_unknown_reference(build_root, capsys): + exit_code = main( + [ + "dev_tools", + "docdiff", + "--build-root", + str(build_root), + "no-such-reference", + "HEAD", + ] + ) + captured = capsys.readouterr() + assert exit_code == 1 + assert "invalid reference" in captured.err + assert not (build_root / "docdiff").exists() + + +@pytest.mark.skipif(not _has_git_repository(), reason="not a git repository") +def test_docdiff_cleanup(build_root): + """Left-overs of an interrupted run are removed.""" + repo = Path(directories.get_source_root()) + root = build_root / "docdiff" + source_root = root / "old" / "source" + subprocess.run( + ["git", "worktree", "add", "--detach", str(source_root), "HEAD"], + cwd=repo, + check=True, + capture_output=True, + ) + (root / "new").mkdir() + + docdiff_app._remove_generated(repo, root) + + assert not root.exists() + worktrees = subprocess.run( + ["git", "worktree", "list"], + cwd=repo, + check=True, + capture_output=True, + text=True, + ).stdout + assert str(source_root) not in worktrees diff --git a/dev_tools/tests/test_reused_concepts.py b/dev_tools/tests/test_reused_concepts.py new file mode 100644 index 0000000000..150d739d29 --- /dev/null +++ b/dev_tools/tests/test_reused_concepts.py @@ -0,0 +1,293 @@ +"""Showing concepts of other classes in the documentation of a NeXus class.""" + +from pathlib import Path +from typing import List +from typing import Optional + +import pytest + +from ..docs import NXClassDocGenerator +from ..globals.errors import NXDLParseError +from ..nxdl import nxdl_schema +from ..nxdl import validate_definition + +_TEMPLATE = """ + +{symbols}{reused} Definition to test the reuse of concepts of other classes. + + The entry. +{content} + +""" + + +def _count_reused(rst: str) -> int: + """Number of concepts marked as reused, excluding the legend.""" + return rst.count("*(reused)*") - 1 + + +@pytest.fixture(scope="module") +def doc_generator(): + return NXClassDocGenerator() + + +@pytest.fixture(scope="module") +def xml_schema(): + return nxdl_schema() + + +@pytest.fixture +def generate_doc(tmp_path, doc_generator, xml_schema, monkeypatch): + """Generate the documentation of a test definition.""" + monkeypatch.setattr("dev_tools.docs.nxdl.get_nxdl_root", lambda: tmp_path) + + def generate( + reuse: Optional[List[str]] = None, + symbols: Optional[List[str]] = None, + content: str = "", + ) -> str: + if reuse: + lines = "".join(f" {line}\n" for line in reuse) + reused = f" \n{lines} \n" + else: + reused = "" + if symbols: + lines = "".join(f" {line}\n" for line in symbols) + symbols = f" \n{lines} \n" + else: + symbols = "" + nxdl_file = Path(tmp_path) / "NXtest_reused_concepts.nxdl.xml" + nxdl_file.write_text( + _TEMPLATE.format(symbols=symbols, reused=reused, content=content) + ) + validate_definition(nxdl_file, xml_schema) + return "".join(doc_generator(nxdl_file)) + + return generate + + +def test_reused_ancestors(generate_doc): + """Groups in between are documented as well, without their documentation.""" + rst = generate_doc(['']) + + assert ":bolditalic:`INSTRUMENT`: (optional) :ref:`NXinstrument`" in rst + assert ":bolditalic:`SOURCE`: (optional) :ref:`NXsource`" in rst + assert "**type**: (optional) :ref:`NX_CHAR `" in rst + assert _count_reused(rst) == 3 + assert ":ref:`⤆ `" in rst + + # only the listed concept is documented + assert "type of radiation source" in rst + assert "Collection of the components of the instrument" not in rst + + +def test_reused_no_children(generate_doc): + rst = generate_doc(['']) + + assert ":bolditalic:`SOURCE`: (optional) :ref:`NXsource`" in rst + assert "**type**" not in rst + + +def test_reused_direct_children(generate_doc): + rst = generate_doc( + [''] + ) + + assert "**component_index**: (optional)" in rst + # the children of NXgeometry/SHAPE are not documented + assert ":bolditalic:`SHAPE`: (optional) :ref:`NXshape`" in rst + assert "**size**" not in rst + + +def test_reused_all_children(generate_doc): + rst = generate_doc( + [''], + symbols=[ + '', + '', + ], + ) + + assert ":bolditalic:`SHAPE`: (optional) :ref:`NXshape`" in rst + assert "**size**: (optional)" in rst + + +def test_reused_too_many_children(generate_doc): + with pytest.raises(NXDLParseError, match="concepts are reused"): + generate_doc(['']) + + +def test_reused_subset_of_children(generate_doc): + rst = generate_doc( + [ + '', + '', + ] + ) + + assert "**type**: (optional)" in rst + assert "**probe**: (optional)" in rst + assert "**name**: (optional)" not in rst + + +def test_reused_attribute(generate_doc): + rst = generate_doc(['']) + + assert "**@default**: (optional) :ref:`NX_CHAR `" in rst + assert ":ref:`⤆ ` *(reused)*" in rst + + +def test_reused_below_defined_group(generate_doc): + """A concept is documented below the group that defines it.""" + content = """ + The instrument. + +""" + rst = generate_doc([''], content=content) + + _, _, below = rst.partition("**instrument**: (required) :ref:`NXinstrument`") + assert ":bolditalic:`SOURCE`: (optional) :ref:`NXsource`" in below + assert _count_reused(rst) == 1 + + +def test_reused_defined_concept(generate_doc): + content = """ + The title. + +""" + with pytest.raises( + NXDLParseError, match="is defined by NXtest_reused_concepts itself" + ): + generate_doc([''], content=content) + + +def test_reused_unknown_concept(generate_doc): + with pytest.raises(NXDLParseError, match="does not exist in"): + generate_doc(['']) + + +def test_reused_renamed_concept(generate_doc): + """A concept that this class redefines under another name is a different one.""" + content = """ + The sample. + +""" + with pytest.raises(NXDLParseError, match="is redefined by .* as 'my_sample'"): + generate_doc([''], content=content) + + # the concept as this class knows it can be reused instead + rst = generate_doc( + [''], content=content + ) + assert "**chemical_formula**: (optional)" in rst + + +def test_reused_invalid_path(generate_doc): + with pytest.raises(NXDLParseError, match="must start with '/'"): + generate_doc(['']) + + +def test_reused_wrong_separator(generate_doc): + with pytest.raises(NXDLParseError, match="is a group"): + generate_doc(['']) + + +def test_reused_listed_twice(generate_doc): + with pytest.raises(NXDLParseError, match="reused more than once"): + generate_doc( + [ + '', + '', + ] + ) + + +def test_symbols_of_reused_concept(generate_doc): + reuse = [''] + + with pytest.raises(NXDLParseError, match="missing from its symbol table"): + generate_doc(reuse) + + rst = generate_doc( + reuse, + symbols=[ + '', + '', + 'Number of columns.', + ], + ) + assert ( + "**nP**: number of scan points (only present in scanning measurements) " + ":ref:`⤆ ` *(reused)*" in rst + ) + assert ".. _/NXtest_reused_concepts/j-symbol:" in rst + assert "**j**: Number of columns." in rst + cls = "/NXtest_reused_concepts" + assert ( + "(Rank: 3, Dimensions: " + f"[:ref:`nP <{cls}/nP-symbol>`, " + f":ref:`i <{cls}/i-symbol>`, " + f":ref:`j <{cls}/j-symbol>`])" in rst + ) + + +def test_symbol_links_in_dimensions(generate_doc): + """Symbols in a dimension size link to the symbol table.""" + content = """ + The data. + + + + + +""" + rst = generate_doc( + content=content, + symbols=['Number of points.'], + ) + + ref = ":ref:`n_data `" + # the escapes keep the inline markup valid next to a digit and a "+" + assert f"(Rank: 2, Dimensions: [2\\ {ref}, {ref}\\ +1])" in rst + + +def test_symbols_of_defined_concept(generate_doc): + content = """ + The data. + + + + +""" + with pytest.raises(NXDLParseError, match="'n_data' \\(dimension of 'data'\\)"): + generate_doc(content=content) + + rst = generate_doc( + symbols=['Number of points.'], + content=content, + ) + assert "**n_data**: Number of points." in rst + + +def test_symbol_reused_from_unknown_class(generate_doc): + with pytest.raises(NXDLParseError, match="unknown class 'NXdoes_not_exist'"): + generate_doc(symbols=['']) + + +def test_symbol_not_in_reused_class(generate_doc): + with pytest.raises(NXDLParseError, match="does not define the symbol 'nP'"): + generate_doc(symbols=['']) + + +def test_symbol_reused_with_doc(generate_doc): + with pytest.raises( + NXDLParseError, match="has a 'doc' element as well as reused_from" + ): + generate_doc( + symbols=[ + 'Points.' + ] + ) diff --git a/manual/source/classes/index.rst b/manual/source/classes/index.rst index 0fc2482c84..b85b296f4b 100644 --- a/manual/source/classes/index.rst +++ b/manual/source/classes/index.rst @@ -14,7 +14,7 @@ application definitions (groupings of objects for a particular technique) and contributed_definitions (proposed definitions from the community) The complete :index:`!vocabulary` of terms used in NeXus NXDL files (names of -groups, fields, attributes, and links) is available for :ref:`download +groups, fields, attributes, links, and symbols) is available for :ref:`download `. .. rubric:: :styleh2:`Base classes` diff --git a/manual/source/defs_intro.rst b/manual/source/defs_intro.rst index dc2fed71bb..f14ddc5547 100755 --- a/manual/source/defs_intro.rst +++ b/manual/source/defs_intro.rst @@ -63,6 +63,8 @@ readbility and comprehension for those whom are new to an NXDL file, the followi guidelines are strongly encouraged: * All symbols used in the application definition are defined in a single ``Symbols`` table. + This is enforced when building the documentation: every symbol that appears as the size + of a dimension must be in the ``Symbols`` table of the same NXDL file. * The :ref:`name ` of a symbol uses camel case without any white space or underscores. examples: @@ -114,6 +116,51 @@ guidelines are strongly encouraged: ... +When a :ref:`reused concept ` has dimensions expressed in symbols +of the class that defines it, those symbols are taken over with the ``reused_from`` +attribute instead of repeating their documentation: + +.. code-block:: xml + :linenos: + + + + + +.. _Design-ReusedConcepts: + +Reusing concepts of other classes +================================= + +A class takes over everything defined by the class it ``extends`` and by the base +classes of the groups it uses, without repeating any of it. Only what a class defines +itself is shown in its documentation. When a group, field or attribute of another +class is worth showing without changing anything about it, list it in the +``reused_concepts`` list instead of repeating its definition: + +.. code-block:: xml + :linenos: + + + + + + + +Each ``path`` starts at the root of the class and spells every name as in the class that +defines it (an attribute is separated from its parent by ``@``, anything else by ``/``). +The groups in between are shown as well, whether they are defined by this class or not. +The ``children`` attribute selects which children of a reused group or field are shown +too: ``none`` (the default), ``direct`` or ``all``. Show a subset of the children by +listing them separately. + +Reused concepts are marked *(reused)* in the documentation and link to the class that +defines them. The class itself is not affected: reusing a concept changes neither the +class nor the validation of data files. Define a concept in the class itself instead +of reusing it whenever anything about it changes, including its documentation. It is +an error to reuse a concept that the class defines itself, or one that does not exist in +the class. + Annotated Structure =================== diff --git a/manual/source/examples/NXwoni.nxdl.xml b/manual/source/examples/NXwoni.nxdl.xml index 7f2c60002e..6be7e39ade 100755 --- a/manual/source/examples/NXwoni.nxdl.xml +++ b/manual/source/examples/NXwoni.nxdl.xml @@ -25,6 +25,14 @@ xmlns="http://definition.nexusformat.org/nxdl/3.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://definition.nexusformat.org/nxdl/3.1 ../nxdl.xsd"> + + + number of wavelengths + + + number of detectors + + Instrument definition for the fictional WONI powder diffractometer at HYNES. diff --git a/nxdl.xsd b/nxdl.xsd index c9f2744739..395f016f15 100755 --- a/nxdl.xsd +++ b/nxdl.xsd @@ -238,10 +238,20 @@ https://stackoverflow.com/a/48980995/1046449 --> + + + + Use a ``reused_concepts`` list to show groups, fields and attributes + that this class takes over from other classes without redefining + them. + + + - In addition to an optional ``symbols`` list, + In addition to an optional ``symbols`` list + and an optional ``reused_concepts`` list, a ``definition`` may contain any of the items allowed in a ``group``. @@ -1216,6 +1226,114 @@ https://stackoverflow.com/a/48980995/1046449 --> + + + + Name of the NeXus class from which the documentation of this + ``symbol`` is taken. Use this when a concept that is reused by + this class (see the ``reused_concepts`` element) has dimensions + expressed in symbols of that class. A ``symbol`` with a + ``reused_from`` attribute must not have a ``doc`` element: + provide a ``doc`` element instead when the meaning of the + symbol differs from the meaning it has in that class. + + + + + + + + + + + + A class takes over everything defined by the class it ``extends`` and by the + base classes of the groups it uses, without repeating any of it. Only what + a class defines itself is shown in its documentation. A ``reused_concepts`` + list points at concepts that a class reuses unchanged but that are worth + showing in its documentation anyway. For example:: + + + + + + + + The documentation of each ``reuse`` path is rendered at its place in the + structure of this class, marked as reused and with a link to the class that + defines it. Groups and fields in between (``INSTRUMENT`` and ``SOURCE`` in + the example above) are shown as well, even when they are neither defined by + this class nor listed here. Nothing else changes: reusing a concept has no + effect on the class or on the validation of data files. + + + + + + + Describe the purpose of this list of reused concepts. + This documentation will go into the manual. + + + + + + + One group, field or attribute of another class to be shown in + the documentation of this class. + + + + + + + Path of the concept, relative to the root of this class and + starting with ``/``. Each name in the path must be spelled as + in the class that defines it (for example + ``/ENTRY/INSTRUMENT/SOURCE/type``). Attributes are separated + from their parent by ``@``. + + It is an error when the concept at this path does not exist in + this class or when this class defines it itself: remove the + definition or remove this ``reuse`` element. + + + + + + + Which children of the reused group or field are shown in the + documentation as well. Show a subset of the children by + listing them separately. Children that every group takes over from + :ref:`NXobject` are never shown, they can only be listed. + + + + + + + + No children are shown (default). + + + + + + + The direct children are shown, but not their children. + + + + + + + All children are shown recursively. + + + + + +